Spaces:
Running
feat(scenario): rob-unexpected-enemy-spawn — surprise 2nd-wave handling (adversarial robustness / military follow-on anchor)
Browse filesWave-6 ROBUSTNESS / adversarial robustness pack: an obvious 4-unit
hunter cluster (Wave 1) closes in from the centre on the agent's
starting vision row; a SECOND wave was PRE-PLACED at a fog corner
(NE on easy/medium, both NE+SE on hard) and closes in by ~tick 1500
in a hunt pattern from a different bearing than Wave 1. Winning
requires defeating BOTH waves (units_killed_gte 7/8/9), keeping the
fact, AND keeping enough force alive (own_units_gte:2) — so a
forward-rush that chases the visible enemies all the way to the fog
corner (leaving Wave 1 to walk past the unguarded base / screen
attrited at long range) LOSES, and an engage-but-don't-overcommit
disposition WINS.
Pre-placed agent force: 3× 2tnk + 2× e3 (5 defenders total) + fact
+ proc; starting_cash 0 and no tent so the model CANNOT train fresh
infantry — the binding decision is RESERVING the starting force.
Pure-e1 defenders cannot survive the opening hunt-bot rush (verified
empirically — melee'd by 4 e1 hunters in ~200 ticks), so the medium-
tank durability + rocket-soldier range mix is required for the
screen to hold the line.
Discrimination (deterministic scripted, no model / no network):
* stall (observe-only) → LOSS on every level + seed (k=0,
all 5 defenders bled out under attrition before tick 5400)
* forward-rush (attack-move to (90,8) every turn) → LOSS on every
level + seed (chase the corner, base unguarded, screen wiped)
* intended engage-no-overcommit (chase closest visible threat
capped at ≤12 cells from fact) → WIN on every level + seed.
Hard tier: 2 agent spawn_point groups (NORTH base y=12 / SOUTH base
y=28) round-robined by seed. Both Wave-2 fog-corner squads (NE y=8 /
SE y=32) always place since enemy actors ignore spawn_point per
CLAUDE.md; the same-latitude surprise hits FIRST and the opposite-
latitude one is the late follow-on, so a single memorised "expect
Wave 2 from NE" disposition cannot generalise.
Benchmark anchors: adversarial robustness (surprise wave handling),
military follow-on wave doctrine, ops incident-after-incident
triage, SC2 second-attack timing.
Registered in tests/test_hard_tier.py::UPGRADED.
17 pack-specific tests + 2 hard-tier curation tests all pass.
|
@@ -0,0 +1,354 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# rob-unexpected-enemy-spawn.yaml
|
| 2 |
+
#
|
| 3 |
+
# REASONING capability — Wave-6 ROBUSTNESS / adversarial robustness:
|
| 4 |
+
# surprise SECOND wave from an UNEXPECTED direction. The agent has
|
| 5 |
+
# an obvious 4-unit hunter cluster (Wave 1) closing in from the centre
|
| 6 |
+
# on its starting vision row; a SECOND wave was PRE-PLACED at a fog
|
| 7 |
+
# corner (NE on easy/medium, both NE+SE on hard) and closes in by
|
| 8 |
+
# ~tick 1500 in a hunt pattern from a different bearing than Wave 1.
|
| 9 |
+
# Winning requires defeating BOTH waves (`units_killed_gte`),
|
| 10 |
+
# keeping the base (`has_building:fact`), AND keeping enough force
|
| 11 |
+
# alive (`own_units_gte:2`). The relevant capability is RESERVING
|
| 12 |
+
# capacity for a follow-on attack — keeping the defender screen
|
| 13 |
+
# centred on the base instead of chasing routers to the corner and
|
| 14 |
+
# leaving the base exposed.
|
| 15 |
+
#
|
| 16 |
+
# Real-world / benchmark anchors:
|
| 17 |
+
# - Adversarial robustness: surprise wave / follow-on attack handling.
|
| 18 |
+
# - Military follow-on wave doctrine ("the battle isn't over when
|
| 19 |
+
# the enemy you see is dead"; second-echelon engagement).
|
| 20 |
+
# - SC2 second-attack timing (the canonical "second push" the
|
| 21 |
+
# defender must reserve forces against).
|
| 22 |
+
# - Operations: incident-after-incident triage (the same operator is
|
| 23 |
+
# immediately re-paged on a fresh incident from a different
|
| 24 |
+
# subsystem — the relevant skill is reserving capacity).
|
| 25 |
+
#
|
| 26 |
+
# DISCRIMINATIONS (no defect, no cheat — every brute/lazy play LOSES):
|
| 27 |
+
#
|
| 28 |
+
# • stall (observe only): the 5 pre-placed defenders are stance:3
|
| 29 |
+
# (AttackAnything) and auto-engage hostiles in range, but with no
|
| 30 |
+
# explicit pursuit they sit out of weapon arc as the hunters
|
| 31 |
+
# converge from the centre. Across ~5000 ticks the static garrison
|
| 32 |
+
# bleeds out under attrition; the `units_killed_gte` bar is never
|
| 33 |
+
# met (kills ≈ 0) AND own_units drops to zero → fail-clause fires
|
| 34 |
+
# (LOSS, not a draw, before tick 5400).
|
| 35 |
+
#
|
| 36 |
+
# • forward-rush (chase the visible enemies all the way to the fog
|
| 37 |
+
# corner): the defender screen attack-moves to the Wave-2 fog
|
| 38 |
+
# corner (90, 8); en route Wave 1 from the centre walks PAST the
|
| 39 |
+
# base (5 e1 hunters bee-line to the agent centroid which is now
|
| 40 |
+
# east of the fact); the corner-bound defenders take attrition
|
| 41 |
+
# bombardment from the converged hunter swarm, and either die out
|
| 42 |
+
# (`own_units_gte:1` fail) or fail to reach the kill bar inside
|
| 43 |
+
# the `after_ticks:5401` deadline → LOSS.
|
| 44 |
+
#
|
| 45 |
+
# • intended prepare-for-2nd-wave: engage closest threat with
|
| 46 |
+
# `attack_move`, but CAP the chase distance so defenders never
|
| 47 |
+
# stray more than ~12 cells from the fact. When the 2nd wave
|
| 48 |
+
# closes in from the fog corner the screen is already pulled
|
| 49 |
+
# back to base-adjacent range and intercepts it before fact
|
| 50 |
+
# contact. Both waves dead (kill bar met), fact alive,
|
| 51 |
+
# own_units_gte:2 satisfied → WIN.
|
| 52 |
+
#
|
| 53 |
+
# ENGINE FACTS (CLAUDE.md / SCENARIO_QUALITY.md):
|
| 54 |
+
# - Bot `hunt` (openra-sim/src/scripted_bot.rs::Hunt): each enemy
|
| 55 |
+
# unit pursues the NEAREST agent target. Hunt-driven e1 cover
|
| 56 |
+
# ~1 cell per 1.8 ticks, so the centre cluster at x=80 reaches
|
| 57 |
+
# the defender outpost at x=14 in ~120 ticks (well before tick
|
| 58 |
+
# 200); the fog-corner Wave-2 squads at (90, 8) reach the base
|
| 59 |
+
# latitude in ~tick 400..1000 depending on path congestion —
|
| 60 |
+
# close to the "~tick 1500" spec target.
|
| 61 |
+
# - The defender screen is 3× 2tnk (medium tank, durable + dual-
|
| 62 |
+
# purpose anti-armor / anti-infantry cannon) + 2× e3 (rocket
|
| 63 |
+
# soldier, range 7, high anti-vehicle DPS). 5× e1 alone get
|
| 64 |
+
# overwhelmed by hunt-bot e1s at melee in the first 200 ticks
|
| 65 |
+
# (verified — see git history); the 2tnk durability is needed
|
| 66 |
+
# to absorb the opening rush, the e3 range is needed to bleed
|
| 67 |
+
# the trailing waves.
|
| 68 |
+
# - Pre-placed defenders are stance:3 (AttackAnything) so the
|
| 69 |
+
# "stall" baseline is genuinely passive (no agent input
|
| 70 |
+
# required for any auto-fire) and the FAILURE MODE on stall is
|
| 71 |
+
# a slow attrition bleed across the whole ~5400-tick budget,
|
| 72 |
+
# not an instant first-turn force-wipe.
|
| 73 |
+
# - Inert enemy `fact` marker far east (anti-DRAW): without a
|
| 74 |
+
# persistent enemy actor the engine auto-`done`s on enemy-elim
|
| 75 |
+
# once both waves die, before the win/fail evaluator runs
|
| 76 |
+
# (CLAUDE.md rule 5).
|
| 77 |
+
# - Tick budget: max_turns 90 with `enemy_unit_spotted` +
|
| 78 |
+
# `own_unit_destroyed` interrupts → in practice ~60..75 ticks
|
| 79 |
+
# per re-decision → reachable tick at max_turns ≈ 5400..6750.
|
| 80 |
+
# `after_ticks: 5401` fires inside this band so a non-winner is
|
| 81 |
+
# a real reachable timeout LOSS, never a draw.
|
| 82 |
+
# - `enemy_unit_spotted` interrupt → re-decide the instant Wave 2
|
| 83 |
+
# becomes visible (faster detection of the surprise axis).
|
| 84 |
+
# - `own_unit_destroyed` interrupt → re-decide when a defender
|
| 85 |
+
# falls.
|
| 86 |
+
# - The pre-placed 5 defenders + fact + proc (no tent / cash 0)
|
| 87 |
+
# means the model CANNOT train fresh infantry — the binding
|
| 88 |
+
# decision is RESERVING the starting force, not out-producing
|
| 89 |
+
# the threat.
|
| 90 |
+
# - hard tier: 2 agent spawn_point groups (NORTH base y=12 /
|
| 91 |
+
# SOUTH base y=28) so the corner the surprise wave comes FROM
|
| 92 |
+
# flips per seed; symmetric Wave-2 squads at BOTH corners
|
| 93 |
+
# always place (enemy actors ignore spawn_point — CLAUDE.md),
|
| 94 |
+
# so a single memorised "expect Wave 2 from NE" disposition
|
| 95 |
+
# cannot generalise.
|
| 96 |
+
|
| 97 |
+
meta:
|
| 98 |
+
id: rob-unexpected-enemy-spawn
|
| 99 |
+
title: 'Surprise Second Wave — Reserve Forces for the Follow-On'
|
| 100 |
+
capability: reasoning
|
| 101 |
+
real_world_meaning: >
|
| 102 |
+
Adversarial robustness on the temporal axis: the obvious threat
|
| 103 |
+
(a 4-unit cluster near the agent's starting vision) is just the
|
| 104 |
+
OPENING; a SECOND wave was pre-placed at a fog corner and closes
|
| 105 |
+
in from an unexpected bearing by ~tick 1500 in a hunt pattern.
|
| 106 |
+
Winning requires destroying BOTH waves AND keeping the base AND
|
| 107 |
+
keeping enough force alive. The relevant capability is RESERVING
|
| 108 |
+
capacity for a follow-on attack — not committing every defender
|
| 109 |
+
to the visible battle and chasing routers far from the base.
|
| 110 |
+
"Battle finished, push deep" is the canonical military / SRE
|
| 111 |
+
failure mode this trains against.
|
| 112 |
+
robotics_analogue: >
|
| 113 |
+
Incident-after-incident triage: a fresh page arrives from a
|
| 114 |
+
different subsystem the moment the operator resolves the visible
|
| 115 |
+
incident; the operator must keep enough on-call capacity in
|
| 116 |
+
reserve to handle the follow-on instead of fully committing
|
| 117 |
+
every responder to the first symptom. Adversarial robustness:
|
| 118 |
+
treating the first observable threat as ground truth and
|
| 119 |
+
over-fitting the response to it leaves the system exposed when
|
| 120 |
+
the adversary stages a second engagement from a different angle.
|
| 121 |
+
benchmark_anchor:
|
| 122 |
+
- 'adversarial robustness: surprise wave handling'
|
| 123 |
+
- 'military follow-on wave doctrine'
|
| 124 |
+
- 'ops: incident-after-incident triage'
|
| 125 |
+
- 'SC2 second-attack timing'
|
| 126 |
+
author: openra-bench
|
| 127 |
+
|
| 128 |
+
base_map: rush-hour-arena
|
| 129 |
+
# Cash 0: the model CANNOT train fresh defenders (no tent and no
|
| 130 |
+
# cash) — the binding decision is RESERVING the starting 5-unit
|
| 131 |
+
# garrison for the surprise 2nd wave, not out-producing the threat.
|
| 132 |
+
starting_cash: 0
|
| 133 |
+
|
| 134 |
+
base:
|
| 135 |
+
agent:
|
| 136 |
+
faction: allies
|
| 137 |
+
# Hunt bot — each enemy unit pursues the nearest agent target. The
|
| 138 |
+
# Wave-2 squad starts FAR from any agent unit at a fog corner; once
|
| 139 |
+
# it closes inside vision the agent's defender centroid pulls it in.
|
| 140 |
+
enemy:
|
| 141 |
+
faction: soviet
|
| 142 |
+
bot_type: hunt
|
| 143 |
+
tools:
|
| 144 |
+
- observe
|
| 145 |
+
- move_units
|
| 146 |
+
- attack_unit
|
| 147 |
+
- attack_move
|
| 148 |
+
- stop
|
| 149 |
+
planning: true
|
| 150 |
+
# Re-decide the instant a fresh enemy unit becomes visible (the
|
| 151 |
+
# surprise-wave detection signal) or a defender is lost.
|
| 152 |
+
interrupts:
|
| 153 |
+
enemy_unit_spotted: true
|
| 154 |
+
own_unit_destroyed: true
|
| 155 |
+
termination:
|
| 156 |
+
max_ticks: 12000
|
| 157 |
+
# Default actor list (overridden per-level). fact + proc at the
|
| 158 |
+
# west base, no tent / no cash — the agent cannot train replacements.
|
| 159 |
+
actors:
|
| 160 |
+
- {type: fact, owner: agent, position: [10, 20]}
|
| 161 |
+
- {type: proc, owner: agent, position: [6, 20]}
|
| 162 |
+
|
| 163 |
+
levels:
|
| 164 |
+
# ── EASY ─────────────────────────────────────────────────────────
|
| 165 |
+
# 2nd wave SMALLER (3 e1) than Wave 1 (4 e1). Kills bar 7 — the
|
| 166 |
+
# full kill-able enemy force on this level — so the policy MUST
|
| 167 |
+
# defeat BOTH waves (a forward chase that leaves the centre
|
| 168 |
+
# cluster to bombard the screen at long range cannot reach kills=7
|
| 169 |
+
# before the screen is wiped). max_turns 90 with interrupts
|
| 170 |
+
# → reachable tick ~5400..6750; `after_ticks: 5401` bites well
|
| 171 |
+
# inside this band so non-win is a reachable LOSS.
|
| 172 |
+
easy:
|
| 173 |
+
description: >
|
| 174 |
+
Adversarial robustness — surprise second wave (easy). An obvious
|
| 175 |
+
4-unit rifle cluster sits at the centre of the arena and closes
|
| 176 |
+
in on your base; a SECOND smaller wave (3 rifles) is pre-placed
|
| 177 |
+
at a fog corner and will close in from an unexpected bearing in
|
| 178 |
+
a hunt pattern by ~tick 1500. Your 5 pre-placed defenders
|
| 179 |
+
(3 medium tanks + 2 rocket soldiers) are your ENTIRE force —
|
| 180 |
+
you have no barracks and no cash so you cannot train
|
| 181 |
+
replacements. To WIN you must, before tick 5400, kill ≥7 enemy
|
| 182 |
+
units across BOTH waves AND still have your construction yard
|
| 183 |
+
(fact) standing AND still have ≥2 of your own units alive.
|
| 184 |
+
Stalling (passive defence — the garrison bleeds out under
|
| 185 |
+
attrition), chasing the visible cluster all the way to the
|
| 186 |
+
fog corner (Wave 1 walks past your base while your screen is
|
| 187 |
+
far afield), or busting the attrition floor (≤3 own units lost)
|
| 188 |
+
all LOSE.
|
| 189 |
+
overrides:
|
| 190 |
+
actors:
|
| 191 |
+
- {type: fact, owner: agent, position: [10, 20]}
|
| 192 |
+
- {type: proc, owner: agent, position: [6, 20]}
|
| 193 |
+
# 5 pre-placed defenders (3× 2tnk + 2× e3), stance:3
|
| 194 |
+
# (AttackAnything — they auto-engage hostiles in weapon arc
|
| 195 |
+
# but do not actively pursue beyond their range). 5× e1 alone
|
| 196 |
+
# cannot survive the opening hunt-bot rush (verified —
|
| 197 |
+
# melee'd in ~200 ticks); the 2tnk durability + e3 reach are
|
| 198 |
+
# both needed for the screen to hold the line.
|
| 199 |
+
- {type: 2tnk, owner: agent, position: [14, 20], stance: 3}
|
| 200 |
+
- {type: 2tnk, owner: agent, position: [14, 18], stance: 3}
|
| 201 |
+
- {type: 2tnk, owner: agent, position: [14, 22], stance: 3}
|
| 202 |
+
- {type: e3, owner: agent, position: [16, 19], stance: 3}
|
| 203 |
+
- {type: e3, owner: agent, position: [16, 21], stance: 3}
|
| 204 |
+
# Wave 1 — obvious 4-unit cluster at the centre on the
|
| 205 |
+
# base latitude. Stance:3 + hunt bot drives them straight
|
| 206 |
+
# at the agent centroid; they arrive at the defender
|
| 207 |
+
# outpost by ~tick 150.
|
| 208 |
+
- {type: e1, owner: enemy, position: [80, 20], stance: 3, count: 4}
|
| 209 |
+
# Wave 2 — SMALLER (3 e1) pre-placed at NE fog corner. Hunt
|
| 210 |
+
# bot closes in from y=8 (out of agent's initial vision row);
|
| 211 |
+
# arrives at base latitude ~tick 600..1000.
|
| 212 |
+
- {type: e1, owner: enemy, position: [90, 8], stance: 3, count: 3}
|
| 213 |
+
# Inert enemy `fact` marker far east (anti-DRAW): keeps the
|
| 214 |
+
# episode alive past the last killed hunter so the win/fail
|
| 215 |
+
# evaluator runs.
|
| 216 |
+
- {type: fact, owner: enemy, position: [120, 20]}
|
| 217 |
+
win_condition:
|
| 218 |
+
all_of:
|
| 219 |
+
- units_killed_gte: 7
|
| 220 |
+
- has_building: fact
|
| 221 |
+
- own_units_gte: 2
|
| 222 |
+
- within_ticks: 5400
|
| 223 |
+
# Non-win must be a real LOSS, not a draw. `after_ticks: 5401`
|
| 224 |
+
# is reachable inside max_turns (in practice ~5400..6750 with
|
| 225 |
+
# interrupts); fact-loss and own-units-zero both fail explicitly.
|
| 226 |
+
# NOTE: WIN requires ≥2 alive (the reserve-capacity test); FAIL
|
| 227 |
+
# fires on full wipe (≥1 alive ensures the episode hasn't ended
|
| 228 |
+
# in total elimination).
|
| 229 |
+
fail_condition:
|
| 230 |
+
any_of:
|
| 231 |
+
- after_ticks: 5401
|
| 232 |
+
- not: {has_building: fact}
|
| 233 |
+
- not: {own_units_gte: 1}
|
| 234 |
+
max_turns: 90
|
| 235 |
+
|
| 236 |
+
# ── MEDIUM ───────────────────────────────────────────────────────
|
| 237 |
+
# +1 controlled variable: 2nd wave EQUAL to first (4 e1) → kills bar 8
|
| 238 |
+
# (the canonical spec value). Same clock, same fail.
|
| 239 |
+
medium:
|
| 240 |
+
description: >
|
| 241 |
+
Adversarial robustness — surprise second wave (medium). An
|
| 242 |
+
obvious 4-unit rifle cluster sits at the centre and closes in
|
| 243 |
+
on your base; a SECOND wave OF EQUAL SIZE (4 rifles) is
|
| 244 |
+
pre-placed at a fog corner and closes in from an unexpected
|
| 245 |
+
bearing in a hunt pattern by ~tick 1500. Your 5 pre-placed
|
| 246 |
+
defenders (3 medium tanks + 2 rocket soldiers) are your ENTIRE
|
| 247 |
+
force — no barracks, no cash. To WIN you must, before tick
|
| 248 |
+
5400, kill ≥8 enemy units across BOTH waves AND still have
|
| 249 |
+
your construction yard (fact) standing AND still have ≥2 of
|
| 250 |
+
your own units alive. Stalling, chasing the visible cluster
|
| 251 |
+
all the way to the fog corner, or attrition-busting all LOSE.
|
| 252 |
+
overrides:
|
| 253 |
+
actors:
|
| 254 |
+
- {type: fact, owner: agent, position: [10, 20]}
|
| 255 |
+
- {type: proc, owner: agent, position: [6, 20]}
|
| 256 |
+
- {type: 2tnk, owner: agent, position: [14, 20], stance: 3}
|
| 257 |
+
- {type: 2tnk, owner: agent, position: [14, 18], stance: 3}
|
| 258 |
+
- {type: 2tnk, owner: agent, position: [14, 22], stance: 3}
|
| 259 |
+
- {type: e3, owner: agent, position: [16, 19], stance: 3}
|
| 260 |
+
- {type: e3, owner: agent, position: [16, 21], stance: 3}
|
| 261 |
+
# Wave 1 — 4-unit cluster (canonical spec count).
|
| 262 |
+
- {type: e1, owner: enemy, position: [80, 20], stance: 3, count: 4}
|
| 263 |
+
# Wave 2 — EQUAL size (4 e1) at NE fog corner.
|
| 264 |
+
- {type: e1, owner: enemy, position: [90, 8], stance: 3, count: 4}
|
| 265 |
+
- {type: fact, owner: enemy, position: [120, 20]}
|
| 266 |
+
win_condition:
|
| 267 |
+
all_of:
|
| 268 |
+
- units_killed_gte: 8
|
| 269 |
+
- has_building: fact
|
| 270 |
+
- own_units_gte: 2
|
| 271 |
+
- within_ticks: 5400
|
| 272 |
+
fail_condition:
|
| 273 |
+
any_of:
|
| 274 |
+
- after_ticks: 5401
|
| 275 |
+
- not: {has_building: fact}
|
| 276 |
+
- not: {own_units_gte: 1}
|
| 277 |
+
max_turns: 90
|
| 278 |
+
|
| 279 |
+
# ── HARD ─────────────────────────────────────────────────────────
|
| 280 |
+
# +1 controlled variable: 2nd wave from TWO fog corners (NE + SE)
|
| 281 |
+
# so the surprise direction flips per seed via 2 agent spawn_point
|
| 282 |
+
# groups (NORTH base y=12 / SOUTH base y=28). Enemy actors ignore
|
| 283 |
+
# spawn_point (CLAUDE.md) — both Wave-2 squads always place. The
|
| 284 |
+
# AGENT spawn round-robins between NORTH and SOUTH, so the SAME-
|
| 285 |
+
# latitude surprise hits FIRST (closer geometry) and the OPPOSITE-
|
| 286 |
+
# latitude one is the late follow-on. The model cannot memorise a
|
| 287 |
+
# single fog-corner heading.
|
| 288 |
+
#
|
| 289 |
+
# Wave-2 squads are 4 e1 each (4 + 4 = 8 wave-2 units + 4 wave-1 =
|
| 290 |
+
# 12 total) and the kill bar is raised to 9 — both waves together
|
| 291 |
+
# produce more than enough kills if the defender screen survives,
|
| 292 |
+
# but a forward chase or attrition bust cannot reach 9 before the
|
| 293 |
+
# screen is wiped. Same clock; same tick-aligned fail.
|
| 294 |
+
hard:
|
| 295 |
+
description: >
|
| 296 |
+
Adversarial robustness — surprise second wave (hard). The
|
| 297 |
+
obvious 4-unit cluster sits at the centre; TWO Wave-2 squads
|
| 298 |
+
are pre-placed at OPPOSITE fog corners (NE and SE) and the
|
| 299 |
+
one at the SAME latitude as your base arrives FIRST while
|
| 300 |
+
the OPPOSITE corner one is the long-haul late follow-on.
|
| 301 |
+
Your base round-robins between a NORTH and a SOUTH latitude
|
| 302 |
+
so which corner the surprise wave arrives from cannot be
|
| 303 |
+
memorised. Your 5 pre-placed defenders (3 medium tanks + 2
|
| 304 |
+
rocket soldiers) are your ENTIRE force — no barracks, no
|
| 305 |
+
cash. To WIN you must, before tick 5400, kill ≥9 enemy units
|
| 306 |
+
across BOTH waves AND still have your construction yard
|
| 307 |
+
(fact) standing AND still have ≥2 of your own units alive.
|
| 308 |
+
Stalling, chasing one fog corner deep, or attrition bust all
|
| 309 |
+
LOSE.
|
| 310 |
+
overrides:
|
| 311 |
+
actors:
|
| 312 |
+
# ── spawn_point 0 — NORTH base (y=12). ─────────────────────
|
| 313 |
+
- {type: fact, owner: agent, position: [10, 12], spawn_point: 0}
|
| 314 |
+
- {type: proc, owner: agent, position: [6, 12], spawn_point: 0}
|
| 315 |
+
- {type: 2tnk, owner: agent, position: [14, 12], stance: 3, spawn_point: 0}
|
| 316 |
+
- {type: 2tnk, owner: agent, position: [14, 10], stance: 3, spawn_point: 0}
|
| 317 |
+
- {type: 2tnk, owner: agent, position: [14, 14], stance: 3, spawn_point: 0}
|
| 318 |
+
- {type: e3, owner: agent, position: [16, 11], stance: 3, spawn_point: 0}
|
| 319 |
+
- {type: e3, owner: agent, position: [16, 13], stance: 3, spawn_point: 0}
|
| 320 |
+
# ── spawn_point 1 — SOUTH base (y=28). ─────────────────────
|
| 321 |
+
- {type: fact, owner: agent, position: [10, 28], spawn_point: 1}
|
| 322 |
+
- {type: proc, owner: agent, position: [6, 28], spawn_point: 1}
|
| 323 |
+
- {type: 2tnk, owner: agent, position: [14, 28], stance: 3, spawn_point: 1}
|
| 324 |
+
- {type: 2tnk, owner: agent, position: [14, 26], stance: 3, spawn_point: 1}
|
| 325 |
+
- {type: 2tnk, owner: agent, position: [14, 30], stance: 3, spawn_point: 1}
|
| 326 |
+
- {type: e3, owner: agent, position: [16, 27], stance: 3, spawn_point: 1}
|
| 327 |
+
- {type: e3, owner: agent, position: [16, 29], stance: 3, spawn_point: 1}
|
| 328 |
+
# Wave 1 — obvious 4-unit cluster at the centre on the mid
|
| 329 |
+
# latitude (y=20). The hunt bot drives them toward the
|
| 330 |
+
# nearest agent target so the cluster naturally engages
|
| 331 |
+
# whichever base is occupied, after a ~one-second detour
|
| 332 |
+
# toward the mid-latitude.
|
| 333 |
+
- {type: e1, owner: enemy, position: [80, 20], stance: 3, count: 4}
|
| 334 |
+
# Wave 2 — TWO 4-unit squads pre-placed at BOTH fog corners
|
| 335 |
+
# (NE y=8 and SE y=32). Enemy actors ignore spawn_point so
|
| 336 |
+
# BOTH always place; whichever corner is at the same
|
| 337 |
+
# latitude as the agent reaches the base FIRST (the EARLY
|
| 338 |
+
# surprise), the OPPOSITE-latitude squad arrives LATER
|
| 339 |
+
# (the LATE follow-on).
|
| 340 |
+
- {type: e1, owner: enemy, position: [90, 8], stance: 3, count: 4}
|
| 341 |
+
- {type: e1, owner: enemy, position: [90, 32], stance: 3, count: 4}
|
| 342 |
+
- {type: fact, owner: enemy, position: [120, 20]}
|
| 343 |
+
win_condition:
|
| 344 |
+
all_of:
|
| 345 |
+
- units_killed_gte: 9
|
| 346 |
+
- has_building: fact
|
| 347 |
+
- own_units_gte: 2
|
| 348 |
+
- within_ticks: 5400
|
| 349 |
+
fail_condition:
|
| 350 |
+
any_of:
|
| 351 |
+
- after_ticks: 5401
|
| 352 |
+
- not: {has_building: fact}
|
| 353 |
+
- not: {own_units_gte: 1}
|
| 354 |
+
max_turns: 90
|
|
@@ -512,6 +512,16 @@ UPGRADED = [
|
|
| 512 |
# vs SW→SE on the south edge) flips per seed and a memorised
|
| 513 |
# single-corner reading order cannot generalise across spawns.
|
| 514 |
"scout-multiple-fog-areas",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 515 |
]
|
| 516 |
|
| 517 |
# Consciously NOT spawn-varied, with the reason (keeps the curation
|
|
|
|
| 512 |
# vs SW→SE on the south edge) flips per seed and a memorised
|
| 513 |
# single-corner reading order cannot generalise across spawns.
|
| 514 |
"scout-multiple-fog-areas",
|
| 515 |
+
# Wave-6 ROBUSTNESS / adversarial robustness seed — surprise 2nd-
|
| 516 |
+
# wave handling (military follow-on wave doctrine / SC2 second-
|
| 517 |
+
# attack timing / ops incident-after-incident triage anchor). Hard
|
| 518 |
+
# tier defines two agent spawn_point groups (NORTH base y=12 /
|
| 519 |
+
# SOUTH base y=28) round-robined by seed; symmetric Wave-2 squads
|
| 520 |
+
# at BOTH fog corners (NE y=8 / SE y=32) always place (enemy
|
| 521 |
+
# actors don't honour spawn_point — CLAUDE.md), so which corner
|
| 522 |
+
# the surprise wave arrives FROM flips per seed and a memorised
|
| 523 |
+
# "expect Wave 2 from NE" disposition cannot generalise.
|
| 524 |
+
"rob-unexpected-enemy-spawn",
|
| 525 |
]
|
| 526 |
|
| 527 |
# Consciously NOT spawn-varied, with the reason (keeps the curation
|
|
@@ -0,0 +1,279 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""rob-unexpected-enemy-spawn scenario family, full loop on Rust.
|
| 2 |
+
|
| 3 |
+
Wave-6 ROBUSTNESS / adversarial robustness. The pack tests handling
|
| 4 |
+
of a SURPRISE 2nd wave that closes in from a fog corner from a
|
| 5 |
+
different bearing than the obvious 4-unit Wave-1 cluster.
|
| 6 |
+
|
| 7 |
+
The win predicate makes BOTH waves load-bearing:
|
| 8 |
+
|
| 9 |
+
* `units_killed_gte:{7|8|9}` ⇒ the threat is actually handled (NOT
|
| 10 |
+
just Wave 1 — the bar requires both); a forward-chase that runs
|
| 11 |
+
out of force before both waves are dead can never satisfy this;
|
| 12 |
+
* `has_building:fact` ⇒ the base survives — chasing Wave 2 deep into
|
| 13 |
+
the fog corner while Wave 1 walks past the unguarded base razes
|
| 14 |
+
the fact;
|
| 15 |
+
* `own_units_gte:2` ⇒ a minimum reserve survived — attrition busts
|
| 16 |
+
(forward-rush, stall) get the screen wiped (≥4 lost from 5 →
|
| 17 |
+
fail);
|
| 18 |
+
* `within_ticks:5400` paired with `after_ticks:5401` ⇒ a non-finisher
|
| 19 |
+
is a real reachable timeout LOSS (90 turns × ≤90 ticks/step
|
| 20 |
+
reaches ≥5400 in interrupt mode), never a draw.
|
| 21 |
+
|
| 22 |
+
These tests prove with deterministic scripted policies (no model,
|
| 23 |
+
no network) that:
|
| 24 |
+
|
| 25 |
+
* the intended engage-but-don't-chase-deep policy WINS every level +
|
| 26 |
+
every hard seed (1..4);
|
| 27 |
+
* stall / forward-rush both LOSE every level + every hard seed (a
|
| 28 |
+
real LOSS, not a draw);
|
| 29 |
+
* the `after_ticks` deadline is reachable inside `max_turns`;
|
| 30 |
+
* the hard tier defines ≥2 spawn_point groups (so the fog corner
|
| 31 |
+
the 2nd wave arrives from varies by seed — single-cell
|
| 32 |
+
memorisation cannot generalise).
|
| 33 |
+
"""
|
| 34 |
+
|
| 35 |
+
from __future__ import annotations
|
| 36 |
+
|
| 37 |
+
import math
|
| 38 |
+
|
| 39 |
+
import pytest
|
| 40 |
+
|
| 41 |
+
pytest.importorskip("openra_train", reason="Rust env wheel not installed")
|
| 42 |
+
pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
|
| 43 |
+
from openra_bench.eval_core import run_level
|
| 44 |
+
from openra_bench.scenarios import load_pack
|
| 45 |
+
from openra_bench.scenarios.loader import PACKS_DIR, compile_level
|
| 46 |
+
|
| 47 |
+
PACK = PACKS_DIR / "rob-unexpected-enemy-spawn.yaml"
|
| 48 |
+
LEVELS = ("easy", "medium", "hard")
|
| 49 |
+
SEEDS = (1, 2, 3, 4)
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
# ── scripted policies ─────────────────────────────────────────────────
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
def _own_ids(rs):
|
| 56 |
+
return [str(u["id"]) for u in (rs.get("units_summary") or [])]
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
def _fact_xy(rs):
|
| 60 |
+
fact = next(
|
| 61 |
+
(b for b in (rs.get("own_buildings") or []) if b.get("type") == "fact"),
|
| 62 |
+
None,
|
| 63 |
+
)
|
| 64 |
+
if fact is None:
|
| 65 |
+
return None
|
| 66 |
+
return int(fact["cell_x"]), int(fact["cell_y"])
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
def _visible_enemy_units(rs):
|
| 70 |
+
"""List of (x, y) for every VISIBLE enemy infantry (not the inert
|
| 71 |
+
fact marker far east)."""
|
| 72 |
+
raw = rs.get("_raw") or {}
|
| 73 |
+
ep = raw.get("enemy_positions") if isinstance(raw, dict) else None
|
| 74 |
+
out = []
|
| 75 |
+
if isinstance(ep, list):
|
| 76 |
+
for e in ep:
|
| 77 |
+
if isinstance(e, dict) and e.get("actor_type") in ("e1", "e3"):
|
| 78 |
+
out.append((int(e.get("cell_x", 0)), int(e.get("cell_y", 0))))
|
| 79 |
+
return out
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
def stall(rs, C):
|
| 83 |
+
"""Observe-only baseline. The 5 defenders are stance:3 so they
|
| 84 |
+
auto-fire on hostiles in weapon arc, but without any pursuit they
|
| 85 |
+
bleed out under attrition across the full 5400-tick budget
|
| 86 |
+
(k=0, all 5 lost) → fail-clause fires (LOSS, not a draw)."""
|
| 87 |
+
return [C.observe()]
|
| 88 |
+
|
| 89 |
+
|
| 90 |
+
def forward_rush(rs, C):
|
| 91 |
+
"""Defeat-first-only-relax taken to its degenerate limit: chase
|
| 92 |
+
the visible enemies all the way to the Wave-2 fog corner (90, 8).
|
| 93 |
+
Wave 1 from the centre walks PAST the base (the agent centroid
|
| 94 |
+
is now far east); the screen takes attrition bombardment from the
|
| 95 |
+
converged hunter swarm and either gets wiped (`own_units_gte:1`
|
| 96 |
+
fail) or fails to meet the kill bar before the `after_ticks:5401`
|
| 97 |
+
deadline → LOSS on every level + seed."""
|
| 98 |
+
own = _own_ids(rs)
|
| 99 |
+
if not own:
|
| 100 |
+
return [C.observe()]
|
| 101 |
+
return [C.attack_move(own, 90, 8)]
|
| 102 |
+
|
| 103 |
+
|
| 104 |
+
def make_intended():
|
| 105 |
+
"""Engage the closest visible threat with `attack_move`, but CAP
|
| 106 |
+
the chase distance so the defender screen never strays more than
|
| 107 |
+
~12 cells from the fact. When the 2nd wave closes in from the
|
| 108 |
+
fog corner the screen is already pulled back to base-adjacent
|
| 109 |
+
range and intercepts before fact contact. Both waves dead, fact
|
| 110 |
+
alive, own_units_gte:2 satisfied → WIN every level + seed."""
|
| 111 |
+
|
| 112 |
+
def policy(rs, C):
|
| 113 |
+
own = _own_ids(rs)
|
| 114 |
+
if not own:
|
| 115 |
+
return [C.observe()]
|
| 116 |
+
xy = _fact_xy(rs)
|
| 117 |
+
if xy is None:
|
| 118 |
+
return [C.attack_move(own, 30, 20)]
|
| 119 |
+
fx, fy = xy
|
| 120 |
+
threats = _visible_enemy_units(rs)
|
| 121 |
+
if threats:
|
| 122 |
+
threats.sort(key=lambda p: math.hypot(p[0] - fx, p[1] - fy))
|
| 123 |
+
ex, ey = threats[0]
|
| 124 |
+
# Reserve-capacity guardrail: don't chase enemies > 12
|
| 125 |
+
# cells from the fact — fall back to a base-adjacent ring.
|
| 126 |
+
if abs(ex - fx) > 12 or abs(ey - fy) > 12:
|
| 127 |
+
return [C.attack_move(own, fx + 6, ey)]
|
| 128 |
+
return [C.attack_move(own, ex, ey)]
|
| 129 |
+
return [C.attack_move(own, fx + 4, fy)]
|
| 130 |
+
|
| 131 |
+
return policy
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
# ── scenario-shape invariants ─────────────────────────────────────────
|
| 135 |
+
|
| 136 |
+
|
| 137 |
+
def test_pack_compiles_with_three_levels_and_hunt_bot():
|
| 138 |
+
pack = load_pack(PACK)
|
| 139 |
+
assert pack.meta.id == "rob-unexpected-enemy-spawn"
|
| 140 |
+
assert pack.meta.capability == "reasoning"
|
| 141 |
+
assert set(pack.levels) == {"easy", "medium", "hard"}
|
| 142 |
+
# Required-by-spec benchmark anchors (the four named in meta).
|
| 143 |
+
anchors = pack.meta.benchmark_anchor
|
| 144 |
+
assert any("adversarial robustness" in a for a in anchors), anchors
|
| 145 |
+
assert any("follow-on" in a for a in anchors), anchors
|
| 146 |
+
assert any("SC2" in a for a in anchors), anchors
|
| 147 |
+
# Hunt bot must be wired through to the engine for every level.
|
| 148 |
+
for lvl in LEVELS:
|
| 149 |
+
c = compile_level(pack, lvl)
|
| 150 |
+
assert c.map_supported
|
| 151 |
+
enemy = c.scenario.enemy
|
| 152 |
+
bot = getattr(enemy, "bot_type", None) or getattr(enemy, "bot", None)
|
| 153 |
+
assert str(bot).lower() == "hunt", (lvl, bot)
|
| 154 |
+
|
| 155 |
+
|
| 156 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 157 |
+
def test_every_level_has_a_reachable_timeout_fail(level):
|
| 158 |
+
"""Non-win must be a real LOSS: the `after_ticks` fail must be
|
| 159 |
+
strictly below the tick reachable at max_turns (≤90 ticks/step in
|
| 160 |
+
interrupt mode)."""
|
| 161 |
+
c = compile_level(load_pack(PACK), level)
|
| 162 |
+
assert c.fail_condition is not None
|
| 163 |
+
fc = c.fail_condition.model_dump(exclude_none=True)
|
| 164 |
+
deadline = None
|
| 165 |
+
for clause in fc.get("any_of", []) or []:
|
| 166 |
+
if "after_ticks" in clause:
|
| 167 |
+
deadline = int(clause["after_ticks"])
|
| 168 |
+
assert deadline is not None, f"{level}: no after_ticks fail clause"
|
| 169 |
+
reachable = 93 + 90 * (c.max_turns - 1)
|
| 170 |
+
assert deadline < reachable, (
|
| 171 |
+
f"{level}: deadline {deadline} unreachable within "
|
| 172 |
+
f"{c.max_turns} turns (max tick {reachable}) → draw degeneracy"
|
| 173 |
+
)
|
| 174 |
+
|
| 175 |
+
|
| 176 |
+
def test_hard_has_two_spawn_point_groups():
|
| 177 |
+
"""Hard-tier contract: ≥2 distinct seed-driven spawn_point groups
|
| 178 |
+
so the fog corner the surprise wave arrives from varies by seed
|
| 179 |
+
(anti-memorisation)."""
|
| 180 |
+
c = compile_level(load_pack(PACK), "hard")
|
| 181 |
+
groups = {
|
| 182 |
+
a.spawn_point for a in c.scenario.actors
|
| 183 |
+
if a.owner == "agent" and a.spawn_point is not None
|
| 184 |
+
}
|
| 185 |
+
assert groups == {0, 1}, groups
|
| 186 |
+
# In-bounds check (rush-hour-arena playable y ≈ 2..38, x ≈ 2..126):
|
| 187 |
+
for a in c.scenario.actors:
|
| 188 |
+
x, y = a.position
|
| 189 |
+
assert 2 <= x <= 126 and 2 <= y <= 38, (a.type, a.position)
|
| 190 |
+
|
| 191 |
+
|
| 192 |
+
def test_starting_cash_zero_for_all_levels():
|
| 193 |
+
"""Reserve-capacity discrimination: no cash + no tent = the model
|
| 194 |
+
CANNOT train fresh defenders, so the binding decision is
|
| 195 |
+
RESERVING the starting force, not out-producing the threat."""
|
| 196 |
+
pack = load_pack(PACK)
|
| 197 |
+
assert pack.starting_cash == 0
|
| 198 |
+
# No barracks pre-placed on any level.
|
| 199 |
+
for lvl in LEVELS:
|
| 200 |
+
c = compile_level(pack, lvl)
|
| 201 |
+
agent_buildings = {
|
| 202 |
+
a.type for a in c.scenario.actors
|
| 203 |
+
if a.owner == "agent" and a.type in ("tent", "barr", "hand")
|
| 204 |
+
}
|
| 205 |
+
assert not agent_buildings, (lvl, agent_buildings)
|
| 206 |
+
|
| 207 |
+
|
| 208 |
+
def test_anti_draw_marker_present_on_every_level():
|
| 209 |
+
"""Anti-DRAW: an unarmed enemy `fact` marker must persist past
|
| 210 |
+
the last hunter death so the engine doesn't auto-`done` on
|
| 211 |
+
enemy-elim before the win/fail evaluator runs (CLAUDE.md rule 5)."""
|
| 212 |
+
pack = load_pack(PACK)
|
| 213 |
+
for lvl in LEVELS:
|
| 214 |
+
c = compile_level(pack, lvl)
|
| 215 |
+
enemy_facts = [
|
| 216 |
+
a for a in c.scenario.actors
|
| 217 |
+
if a.owner == "enemy" and a.type == "fact"
|
| 218 |
+
]
|
| 219 |
+
assert len(enemy_facts) >= 1, lvl
|
| 220 |
+
|
| 221 |
+
|
| 222 |
+
# ── solvency: intended WINS every level + every hard seed ────────────
|
| 223 |
+
|
| 224 |
+
|
| 225 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 226 |
+
def test_intended_engage_no_overcommit_wins_every_level_and_seed(level):
|
| 227 |
+
c = compile_level(load_pack(PACK), level)
|
| 228 |
+
for seed in SEEDS:
|
| 229 |
+
r = run_level(c, make_intended(), seed=seed)
|
| 230 |
+
assert r.outcome == "win", (
|
| 231 |
+
f"{level} seed{seed}: intended engage-no-overcommit play "
|
| 232 |
+
f"must WIN; got {r.outcome} (tick={r.signals.game_tick}, "
|
| 233 |
+
f"kills={r.signals.units_killed}, "
|
| 234 |
+
f"lost={r.signals.units_lost}, "
|
| 235 |
+
f"buildings={r.signals.own_buildings})"
|
| 236 |
+
)
|
| 237 |
+
|
| 238 |
+
|
| 239 |
+
# ── no-cheat: every lazy / brute policy LOSES (not draws) ────────────
|
| 240 |
+
|
| 241 |
+
|
| 242 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 243 |
+
@pytest.mark.parametrize(
|
| 244 |
+
"policy_name,policy_factory",
|
| 245 |
+
[
|
| 246 |
+
("stall", lambda: stall),
|
| 247 |
+
("forward_rush", lambda: forward_rush),
|
| 248 |
+
],
|
| 249 |
+
)
|
| 250 |
+
def test_lazy_and_overcommit_policies_lose_every_level_and_seed(
|
| 251 |
+
level, policy_name, policy_factory
|
| 252 |
+
):
|
| 253 |
+
"""Stall (passive — bleeds out under attrition) and forward-rush
|
| 254 |
+
(over-commits to the fog corner — base unguarded / screen wiped)
|
| 255 |
+
must BOTH LOSE on every level + every seed — no draw."""
|
| 256 |
+
c = compile_level(load_pack(PACK), level)
|
| 257 |
+
for seed in SEEDS:
|
| 258 |
+
r = run_level(c, policy_factory(), seed=seed)
|
| 259 |
+
assert r.outcome == "loss", (
|
| 260 |
+
f"{level} seed{seed} {policy_name}: must LOSE (real fail, "
|
| 261 |
+
f"not a draw); got {r.outcome} (tick={r.signals.game_tick}, "
|
| 262 |
+
f"kills={r.signals.units_killed}, "
|
| 263 |
+
f"lost={r.signals.units_lost}, "
|
| 264 |
+
f"buildings={r.signals.own_buildings})"
|
| 265 |
+
)
|
| 266 |
+
|
| 267 |
+
|
| 268 |
+
# ── determinism ───────────────────────────────────────────────────────
|
| 269 |
+
|
| 270 |
+
|
| 271 |
+
def test_intended_run_is_deterministic_on_easy():
|
| 272 |
+
c = compile_level(load_pack(PACK), "easy")
|
| 273 |
+
a = run_level(c, make_intended(), seed=3)
|
| 274 |
+
b = run_level(c, make_intended(), seed=3)
|
| 275 |
+
assert (a.outcome, a.turns, a.signals.units_killed) == (
|
| 276 |
+
b.outcome,
|
| 277 |
+
b.turns,
|
| 278 |
+
b.signals.units_killed,
|
| 279 |
+
), "same seed must be deterministic"
|