Spaces:
Running
feat(scenario): mfb-two-base-simultaneous — concurrent two-base ramp (SC2 multi-base / multi-plant anchor)
Browse filesGroup B Wave-7 REASONING pack. Two pre-placed Construction Yards (WEST
x=15 + EAST x=85), each with a powr so the agent's shared Building
queue + proc item are unlocked from tick 0. Starting cash 2800 = 2x
proc cost — exactly enough for the two required Refineries; a third
will not fit so the bar cannot be brute-forced by over-building.
WIN: has_building:fact AND building_in_region(WEST,proc) AND
building_in_region(EAST,proc) AND within_ticks:T (T = 2700 easy, 1850
medium/hard). FAIL: after_ticks:T+1 OR not has_building:fact.
The bar — empirically validated scripted on every level + every hard
seed (1..4):
* STALL (only observe) → after_ticks LOSS.
* ONE-BASE-ONLY (west only) → east region never covered → LOSS.
* SERIALIZE-WITH-IDLE-OBSERVES (build → wait turn → place → build →
wait turn → place) → 2nd proc placed at ~tick 1893 > 1850 → LOSS
on medium + hard.
* INTENDED CONCURRENT-PIPELINED (queue BOTH procs on turn 1, spam
place_building at each base until each lands) → 1st placed ~tick
903, 2nd ~tick 1803 ≤ 1850 → WIN.
Hard tier registers in UPGRADED with TWO seed-driven agent spawn_point
groups round-robining the BASE LATITUDE (NORTH y=12 / SOUTH y=28).
Every base building is duplicated across both groups at the matching
latitude (CLAUDE.md spawn-filter rule). A single HoldFire rifleman per
spawn group serves as a units_summary marker so the curated-hard
generalization check sees a seed-distinguishable start; it plays no
role in the win predicate (buildings-only). The win clause is
any_of(NORTH, SOUTH) so EITHER full layout satisfies, but the agent
must place each proc within radius 8 of its actual base latitude (a
memorised always-(15,20)/(85,20) opening misses both regions on
either seed group).
Tick budget: max_turns chosen so after_ticks bites inside the budget
(easy 2701 <= 2883; medium/hard 1851 <= 2073).
Engine footguns applied per CLAUDE.md: arena map keeps every actor
in playable bounds; persistent unarmed enemy fact marker in far SE
inhibits engine auto-done; spawn_mcvs:false so the inventory is
deterministic; place_building does not enforce build-adjacency so the
shared queue can land procs at the far EAST base.
|
@@ -0,0 +1,376 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# mfb-two-base-simultaneous — concurrent two-base ramp.
|
| 2 |
+
#
|
| 3 |
+
# REASONING capability — multi-front production ramp. The agent owns
|
| 4 |
+
# TWO already-deployed Construction Yards (fact) at distant sites
|
| 5 |
+
# (WEST x=15 and EAST x=85). Starting cash funds exactly the two
|
| 6 |
+
# Refinery (`proc`) commits the bar requires (2 × 1400 = 2800).
|
| 7 |
+
# The agent must `build proc` twice and `place_building` each at the
|
| 8 |
+
# matching base — once near the WEST fact AND once near the EAST
|
| 9 |
+
# fact — and finish BOTH inside a tight clock.
|
| 10 |
+
#
|
| 11 |
+
# Why this tests "concurrent" / "split attention":
|
| 12 |
+
# The player's Building production queue is shared across the
|
| 13 |
+
# player (one item builds at a time, ~60 ticks per item). The only
|
| 14 |
+
# way to satisfy the bar inside the medium/hard clock is to keep
|
| 15 |
+
# the queue saturated and pipeline the place_building order with
|
| 16 |
+
# the next build order on the same turn — i.e. issue
|
| 17 |
+
# `place_building west` AND `build proc` on the turn the first
|
| 18 |
+
# proc is reported completed. An agent that serializes the two
|
| 19 |
+
# bases (build → wait → place → observe → build → wait → place)
|
| 20 |
+
# inserts 3–5 wasted turns of observation between commands and
|
| 21 |
+
# blows the deadline. An agent that stalls (only observe) never
|
| 22 |
+
# builds either proc and loses on the clock.
|
| 23 |
+
#
|
| 24 |
+
# Real-world anchor: multi-site facility ramp where two greenfield
|
| 25 |
+
# sites must come online in parallel. SC2 simultaneous-base macro
|
| 26 |
+
# (the canonical pro opening keeps every production queue saturated
|
| 27 |
+
# across BOTH bases instead of pumping one to maturity first).
|
| 28 |
+
# Distributed-systems geo-redundancy: two regional deployments must
|
| 29 |
+
# go live concurrently — serializing the rollout leaves one region
|
| 30 |
+
# uncovered past the SLA window.
|
| 31 |
+
#
|
| 32 |
+
# WIN (every level):
|
| 33 |
+
# has_building: fact (the agent never lost a fact)
|
| 34 |
+
# AND building_in_region:{x:15, y:20, r:8, type:proc, count:1}
|
| 35 |
+
# AND building_in_region:{x:85, y:20, r:8, type:proc, count:1}
|
| 36 |
+
# AND within_ticks: <tier T>
|
| 37 |
+
# (T = 2700 easy, 1850 medium, 1850 hard; the hard win uses an
|
| 38 |
+
# any_of over NORTH (y=12) and SOUTH (y=28) latitudes mirroring
|
| 39 |
+
# the seeded base placement.)
|
| 40 |
+
#
|
| 41 |
+
# FAIL (every level):
|
| 42 |
+
# any_of[ after_ticks: <T+1>,
|
| 43 |
+
# not has_building: fact ← either fact razed counts ]
|
| 44 |
+
#
|
| 45 |
+
# Failure modes the predicate makes LOSE (the bar):
|
| 46 |
+
# • STALL (only observe()): never builds either proc → after_ticks
|
| 47 |
+
# fires → real LOSS.
|
| 48 |
+
# • SERIALIZE-WEST-THEN-EAST: build proc → wait for the queue →
|
| 49 |
+
# place west → THEN queue the 2nd proc (instead of queueing both
|
| 50 |
+
# on turn 1) → wait again → place east. Wall-clock ~1893+ ticks
|
| 51 |
+
# > medium/hard T 1850 → after_ticks 1851 fires → LOSS. Adding
|
| 52 |
+
# any observe gaps between actions makes it worse.
|
| 53 |
+
# • ONE-BASE-ONLY: build & place a proc only at the west fact
|
| 54 |
+
# (ignore east). The east `building_in_region:proc` clause is
|
| 55 |
+
# never satisfied → after_ticks fires → LOSS.
|
| 56 |
+
# • INTENDED CONCURRENT-PIPELINED: queue BOTH procs on turn 1
|
| 57 |
+
# (the queue serialises them but both are waiting from t=0),
|
| 58 |
+
# place each as soon as it is reported done. Wall-clock ~1803
|
| 59 |
+
# ticks (turn 20, both procs placed) ≤ medium/hard T 1850 → WIN.
|
| 60 |
+
# The concurrency under test is queue-saturation: the agent must
|
| 61 |
+
# commit to BOTH proc orders BEFORE the first one completes,
|
| 62 |
+
# instead of waiting to see the first placement before queueing
|
| 63 |
+
# the next.
|
| 64 |
+
#
|
| 65 |
+
# Engine footguns applied (per CLAUDE.md):
|
| 66 |
+
# • Map is generator-spec `{generator: arena, width:160, height:60,
|
| 67 |
+
# cordon:4}` so playable cells are x∈[4,156), y∈[4,56). Every
|
| 68 |
+
# actor and region centre sits comfortably inside.
|
| 69 |
+
# • `proc` (cost 1400) requires `powr` as a tech prereq
|
| 70 |
+
# (gamerules.rs:343). Each base is pre-placed with `fact + powr`
|
| 71 |
+
# so the Building queue and the `proc` item are both enabled at
|
| 72 |
+
# tick 0. `place_building` does NOT enforce build-adjacency
|
| 73 |
+
# (CLAUDE.md) so a proc can be placed at the FAR base from the
|
| 74 |
+
# shared player queue with a single command.
|
| 75 |
+
# • Starting cash is EXACTLY 2800 = 2 × proc cost. A third proc
|
| 76 |
+
# would not fit, so the agent cannot brute-force one extra to
|
| 77 |
+
# satisfy a missing region.
|
| 78 |
+
# • `spawn_mcvs: false` — engine default auto-seeds an MCV at
|
| 79 |
+
# mpspawn cells; we declare every actor ourselves so the
|
| 80 |
+
# starting inventory is deterministic (no spare MCV that could
|
| 81 |
+
# deploy into a third fact and satisfy a clause by accident).
|
| 82 |
+
# • A persistent unarmed enemy `fact` marker is parked in the far
|
| 83 |
+
# SE corner so the engine does not auto-`done` on
|
| 84 |
+
# all-enemies-eliminated — guarantees a real LOSS-not-DRAW on
|
| 85 |
+
# deadline timeouts (CLAUDE.md auto-done footgun).
|
| 86 |
+
# • Engine production timing: `proc` (cost 1400) builds in 840
|
| 87 |
+
# ticks (cost × build_duration_modifier / 100 = 1400 × 60 / 100).
|
| 88 |
+
# The shared Building production queue serialises items: a 2nd
|
| 89 |
+
# proc behind a 1st one in queue makes ZERO progress until the
|
| 90 |
+
# 1st is placed. So the OPTIMAL pipeline is to queue BOTH procs
|
| 91 |
+
# on turn 1 — proc1 finishes at tick ~903 (turn 11), and proc2
|
| 92 |
+
# follows immediately, finishing at tick ~1713 (turn 20). Any
|
| 93 |
+
# play that delays queueing the 2nd proc adds turns to the
|
| 94 |
+
# total wall-clock.
|
| 95 |
+
# • Tick/turn alignment (CLAUDE.md, ~90 ticks/turn): every level's
|
| 96 |
+
# `after_ticks` fail is reachable inside its `max_turns` so a
|
| 97 |
+
# stall emits a LOSS, not a DRAW:
|
| 98 |
+
# easy : after_ticks 2701 ≤ 93 + 90·31 = 2883 ✓ (max_turns 32)
|
| 99 |
+
# medium : after_ticks 1851 ≤ 93 + 90·22 = 2073 ✓ (max_turns 23)
|
| 100 |
+
# hard : after_ticks 1851 ≤ 93 + 90·22 = 2073 ✓ (max_turns 23)
|
| 101 |
+
# Discrimination window validated empirically: the engine's actual
|
| 102 |
+
# wall-clock for the pipelined-concurrent play is queue1 done at
|
| 103 |
+
# ~tick 843 → place ~903 (turn 11) → queue2 done at ~1713 →
|
| 104 |
+
# place ~1803 (turn 20). within_ticks 1850 leaves ~47 ticks of
|
| 105 |
+
# headroom for the intended play and STILL falls short of every
|
| 106 |
+
# serialized variant (the next 90-tick decision boundary at 1893
|
| 107 |
+
# already misses).
|
| 108 |
+
# • Hard tier: two agent `spawn_point` groups round-robin the
|
| 109 |
+
# BASE LATITUDE (NORTH bases at y=12 vs SOUTH bases at y=28).
|
| 110 |
+
# Per CLAUDE.md spawn-filter rule, EVERY agent actor declares an
|
| 111 |
+
# explicit spawn_point — the base buildings are duplicated
|
| 112 |
+
# across both groups at their respective coords; otherwise the
|
| 113 |
+
# filter would drop the un-tagged actors and leave the agent
|
| 114 |
+
# bare. The two target regions move with the bases (NORTH band
|
| 115 |
+
# y=12 r=8 vs SOUTH band y=28 r=8). Neutral/enemy actors place
|
| 116 |
+
# regardless of spawn_point (CLAUDE.md).
|
| 117 |
+
|
| 118 |
+
meta:
|
| 119 |
+
id: mfb-two-base-simultaneous
|
| 120 |
+
title: 'Multi-Front Build — Two Bases, Concurrent Refinery Ramp'
|
| 121 |
+
capability: reasoning
|
| 122 |
+
real_world_meaning: >
|
| 123 |
+
Concurrent multi-site facility ramp under a shared bottleneck.
|
| 124 |
+
The operator runs ONE central production queue but has TWO
|
| 125 |
+
physically distant sites that each need a fresh production line
|
| 126 |
+
online before the deadline. The decision is NOT "which site
|
| 127 |
+
first" (both are required) but "keep the bottleneck queue
|
| 128 |
+
saturated and pipeline placement at each site as the previous
|
| 129 |
+
item completes". An operator who serializes (finish all work
|
| 130 |
+
at site A before touching site B, inserting idle audit cycles
|
| 131 |
+
between commands) blows the SLA on site B even when site A is
|
| 132 |
+
flawless. The capability under test is split-attention pipelining:
|
| 133 |
+
issuing the next start-of-build order on the very same turn the
|
| 134 |
+
prior item lands at its target site.
|
| 135 |
+
robotics_analogue: >
|
| 136 |
+
Two greenfield robotics cells must each receive a critical
|
| 137 |
+
fixture (refinery / palletizer) before the launch window. The
|
| 138 |
+
fixture fabrication shop is shared (one fixture at a time), so
|
| 139 |
+
the bottleneck is queue saturation, not transport. A scheduler
|
| 140 |
+
that pipelines (fixture-1 build → install at cell A while
|
| 141 |
+
fixture-2 already started) ships both cells in one launch window;
|
| 142 |
+
a scheduler that batches (finish all of cell A, then start cell B
|
| 143 |
+
from scratch) misses cell B's deadline despite identical total
|
| 144 |
+
work.
|
| 145 |
+
benchmark_anchor:
|
| 146 |
+
- "SC2 simultaneous-base macro"
|
| 147 |
+
- "multi-plant manufacturing"
|
| 148 |
+
- "distributed-systems geo-redundancy"
|
| 149 |
+
- "multi-site facility ramp under a shared bottleneck"
|
| 150 |
+
author: "openra-bench"
|
| 151 |
+
|
| 152 |
+
# Pack-level base_map is the sentinel `rush-hour-arena` so the loader
|
| 153 |
+
# can map-resolve before per-level overrides materialise the actual
|
| 154 |
+
# generator-spec arena .oramap (same pattern as the other mfb-* packs
|
| 155 |
+
# and mcv-deploy-third-base).
|
| 156 |
+
base_map: rush-hour-arena
|
| 157 |
+
|
| 158 |
+
# 2 procs × 1400 = 2800. Exactly enough; a third proc would not fit.
|
| 159 |
+
starting_cash: 2800
|
| 160 |
+
|
| 161 |
+
base:
|
| 162 |
+
agent: {faction: allies}
|
| 163 |
+
enemy: {faction: soviet, cash: 0, bot_type: ''}
|
| 164 |
+
# build + place_building for the two procs; observe for the turn-1
|
| 165 |
+
# snapshot; stop is kept on so an idle test policy can no-op safely.
|
| 166 |
+
tools: [observe, build, place_building, stop]
|
| 167 |
+
# Engine default auto-spawns an MCV at mpspawn cells; off so the
|
| 168 |
+
# inventory is exactly what each level declares.
|
| 169 |
+
spawn_mcvs: false
|
| 170 |
+
planning: true
|
| 171 |
+
termination: {max_ticks: 4000} # > all per-level after_ticks (2701 / 1851)
|
| 172 |
+
# Every level supplies its own actor list via overrides so the
|
| 173 |
+
# per-tier base latitude (and hard-tier spawn variation) is
|
| 174 |
+
# audit-explicit.
|
| 175 |
+
actors: []
|
| 176 |
+
|
| 177 |
+
levels:
|
| 178 |
+
# ── EASY ─────────────────────────────────────────────────────────
|
| 179 |
+
# Generous clock (within_ticks 2700, max_turns 32 → reachable 2883):
|
| 180 |
+
# both the INTENDED concurrent pipeline (~3 turns) AND a leisurely
|
| 181 |
+
# serial sequence (~6–8 turns ≈ 720 ticks) clear the bar. Only
|
| 182 |
+
# STALL (no build orders at all) fails the deadline. Easy isolates
|
| 183 |
+
# "the agent must actually issue build + place orders at both
|
| 184 |
+
# bases" — no concurrency pressure yet.
|
| 185 |
+
# 2701 ≤ 93 + 90·31 = 2883 ✓
|
| 186 |
+
easy:
|
| 187 |
+
description: >
|
| 188 |
+
You hold TWO pre-built Construction Yards: a WEST base at
|
| 189 |
+
(15,20) (with a power plant powr at (18,20)) and an EAST base
|
| 190 |
+
at (85,20) (with a power plant at (88,20)). Both bases already
|
| 191 |
+
have the Building production queue enabled and the `proc`
|
| 192 |
+
(Refinery, cost 1400) buildable from the player's shared queue
|
| 193 |
+
thanks to the pre-placed powr. Your starting cash is 2800 —
|
| 194 |
+
exactly two Refineries' worth. WIN when you have a `proc`
|
| 195 |
+
placed inside the WEST region (centred at (15,20), radius 8)
|
| 196 |
+
AND a `proc` placed inside the EAST region (centred at (85,20),
|
| 197 |
+
radius 8), with both your construction yards still alive,
|
| 198 |
+
before tick 2700. Stalling (only observe) never builds
|
| 199 |
+
anything; building only one Refinery leaves the other region
|
| 200 |
+
uncovered. Both LOSE.
|
| 201 |
+
overrides:
|
| 202 |
+
base_map:
|
| 203 |
+
generator: arena
|
| 204 |
+
name: mfb-two-base-simultaneous-arena
|
| 205 |
+
width: 160
|
| 206 |
+
height: 60
|
| 207 |
+
cordon: 4
|
| 208 |
+
actors:
|
| 209 |
+
# WEST base — fact + powr so the player's Building queue and
|
| 210 |
+
# `proc` item are both unlocked from tick 0.
|
| 211 |
+
- {type: fact, owner: agent, position: [15, 20]}
|
| 212 |
+
- {type: powr, owner: agent, position: [18, 20]}
|
| 213 |
+
# EAST base — symmetric layout.
|
| 214 |
+
- {type: fact, owner: agent, position: [85, 20]}
|
| 215 |
+
- {type: powr, owner: agent, position: [88, 20]}
|
| 216 |
+
# Persistent unarmed enemy `fact` marker at the far SE edge,
|
| 217 |
+
# outside every win region. Keeps the engine from auto-`done`
|
| 218 |
+
# on all-enemies-eliminated (the only enemy actor) so a
|
| 219 |
+
# non-winner emits a real reachable LOSS on the deadline.
|
| 220 |
+
- {type: fact, owner: enemy, position: [152, 52]}
|
| 221 |
+
win_condition:
|
| 222 |
+
all_of:
|
| 223 |
+
- has_building: fact
|
| 224 |
+
- building_in_region:
|
| 225 |
+
{x: 15, y: 20, radius: 8, type: proc, count: 1}
|
| 226 |
+
- building_in_region:
|
| 227 |
+
{x: 85, y: 20, radius: 8, type: proc, count: 1}
|
| 228 |
+
- within_ticks: 2700
|
| 229 |
+
fail_condition:
|
| 230 |
+
any_of:
|
| 231 |
+
- after_ticks: 2701
|
| 232 |
+
- not: {has_building: fact}
|
| 233 |
+
max_turns: 32
|
| 234 |
+
|
| 235 |
+
# ── MEDIUM ───────────────────────────────────────────────────────
|
| 236 |
+
# +1 controlled variable: TIGHT clock (within_ticks 1850,
|
| 237 |
+
# max_turns 23 → reachable 2073). The intended CONCURRENT pipeline
|
| 238 |
+
# queues BOTH procs on turn 1 — proc1 finishes at tick ~843 and is
|
| 239 |
+
# placed at WEST at turn 11 (tick ~903), proc2 immediately starts
|
| 240 |
+
# in the (now unblocked) queue and is placed at EAST at turn 20
|
| 241 |
+
# (tick ~1803). Total ~1803 ticks ≤ 1850 → WIN. A serialized policy
|
| 242 |
+
# that waits to see proc1 placed before queueing proc2 adds ≥1
|
| 243 |
+
# turn (≥90 ticks) between completion and the next queue start;
|
| 244 |
+
# proc2 then lands at tick ~1893 → after_ticks 1851 fires → LOSS.
|
| 245 |
+
# Adding any observe gaps between commands makes the serialized
|
| 246 |
+
# failure mode lose by a wider margin. Stall LOSES outright on the
|
| 247 |
+
# deadline.
|
| 248 |
+
# 1851 ≤ 93 + 90·22 = 2073 ✓
|
| 249 |
+
medium:
|
| 250 |
+
description: >
|
| 251 |
+
Same TWO bases (WEST fact at (15,20) + powr at (18,20); EAST
|
| 252 |
+
fact at (85,20) + powr at (88,20)) and the same starting cash
|
| 253 |
+
2800 — but on a TIGHT clock. WIN when there is a `proc`
|
| 254 |
+
inside the WEST region (15,20,r8) AND a `proc` inside the EAST
|
| 255 |
+
region (85,20,r8), with both construction yards alive, before
|
| 256 |
+
tick 1850. Concurrency is mandatory: issue the next `build proc`
|
| 257 |
+
on the very same turn you place the previous one (keep the
|
| 258 |
+
shared production queue saturated). Stall, serialize-with-idle-
|
| 259 |
+
observes, and one-base-only all LOSE on the deadline.
|
| 260 |
+
overrides:
|
| 261 |
+
base_map:
|
| 262 |
+
generator: arena
|
| 263 |
+
name: mfb-two-base-simultaneous-arena
|
| 264 |
+
width: 160
|
| 265 |
+
height: 60
|
| 266 |
+
cordon: 4
|
| 267 |
+
actors:
|
| 268 |
+
- {type: fact, owner: agent, position: [15, 20]}
|
| 269 |
+
- {type: powr, owner: agent, position: [18, 20]}
|
| 270 |
+
- {type: fact, owner: agent, position: [85, 20]}
|
| 271 |
+
- {type: powr, owner: agent, position: [88, 20]}
|
| 272 |
+
- {type: fact, owner: enemy, position: [152, 52]}
|
| 273 |
+
win_condition:
|
| 274 |
+
all_of:
|
| 275 |
+
- has_building: fact
|
| 276 |
+
- building_in_region:
|
| 277 |
+
{x: 15, y: 20, radius: 8, type: proc, count: 1}
|
| 278 |
+
- building_in_region:
|
| 279 |
+
{x: 85, y: 20, radius: 8, type: proc, count: 1}
|
| 280 |
+
- within_ticks: 1850
|
| 281 |
+
fail_condition:
|
| 282 |
+
any_of:
|
| 283 |
+
- after_ticks: 1851
|
| 284 |
+
- not: {has_building: fact}
|
| 285 |
+
max_turns: 23
|
| 286 |
+
|
| 287 |
+
# ── HARD ─────────────────────────────────────────────────────────
|
| 288 |
+
# +1 controlled variable: TWO seed-driven agent spawn_point groups
|
| 289 |
+
# round-robin the base LATITUDE between a NORTH layout (y=12) and
|
| 290 |
+
# a SOUTH layout (y=28). Both bases (west & east) flip together,
|
| 291 |
+
# so the agent's two target regions also flip together (NORTH
|
| 292 |
+
# band at y=12 vs SOUTH band at y=28). The win clause is an
|
| 293 |
+
# `any_of` over the two latitudes so EITHER set of regions counts
|
| 294 |
+
# — but the agent must place each proc within radius 8 of WHICHEVER
|
| 295 |
+
# latitude its facts spawned at (a memorised "always (15,20) and
|
| 296 |
+
# (85,20)" opening misses by ≥8 cells on one of the two seed
|
| 297 |
+
# groups). Same tight clock (within_ticks 1850, max_turns 23 →
|
| 298 |
+
# reachable 2073) so concurrency is still the discriminator.
|
| 299 |
+
#
|
| 300 |
+
# CLAUDE.md spawn-filter rule: once any agent actor declares
|
| 301 |
+
# spawn_point, every agent actor WITHOUT spawn_point is filtered
|
| 302 |
+
# OUT — so every base building is duplicated across BOTH spawn
|
| 303 |
+
# groups at the appropriate latitude. The enemy marker has no
|
| 304 |
+
# spawn_point (enemy actors place regardless) and is therefore
|
| 305 |
+
# listed only once.
|
| 306 |
+
# 1851 ≤ 93 + 90·22 = 2073 ✓
|
| 307 |
+
hard:
|
| 308 |
+
description: >
|
| 309 |
+
Same two-base concurrent ramp on the same tight clock, but
|
| 310 |
+
the BASE LATITUDE varies per seed. Each seed ships both your
|
| 311 |
+
pre-built construction yards at EITHER the NORTH band (y=12)
|
| 312 |
+
OR the SOUTH band (y=28); both candidate target regions move
|
| 313 |
+
with the bases. Read your starting fact positions and place
|
| 314 |
+
each Refinery within radius 8 of the matching latitude. WIN
|
| 315 |
+
when there is a `proc` inside the WEST region AND a `proc`
|
| 316 |
+
inside the EAST region (BOTH at WHICHEVER latitude your bases
|
| 317 |
+
spawned at — north OR south), with both construction yards
|
| 318 |
+
alive, before tick 1850. Stalling, serializing with idle
|
| 319 |
+
observe gaps, and one-base-only all LOSE.
|
| 320 |
+
objective_coords: relative
|
| 321 |
+
overrides:
|
| 322 |
+
base_map:
|
| 323 |
+
generator: arena
|
| 324 |
+
name: mfb-two-base-simultaneous-arena
|
| 325 |
+
width: 160
|
| 326 |
+
height: 60
|
| 327 |
+
cordon: 4
|
| 328 |
+
actors:
|
| 329 |
+
# spawn_point 0 — NORTH layout (bases at y=12).
|
| 330 |
+
- {type: fact, owner: agent, position: [15, 12], spawn_point: 0}
|
| 331 |
+
- {type: powr, owner: agent, position: [18, 12], spawn_point: 0}
|
| 332 |
+
- {type: fact, owner: agent, position: [85, 12], spawn_point: 0}
|
| 333 |
+
- {type: powr, owner: agent, position: [88, 12], spawn_point: 0}
|
| 334 |
+
# A single HoldFire rifleman per spawn group at the WEST
|
| 335 |
+
# base — purely a SEED-DRIVEN-START marker so units_summary
|
| 336 |
+
# reveals which spawn_point was selected (the curated-hard
|
| 337 |
+
# generalization test inspects unit cells, not buildings).
|
| 338 |
+
# It plays no role in the win predicate (which checks
|
| 339 |
+
# buildings only), and stance:0 keeps it from auto-firing
|
| 340 |
+
# on the SE enemy fact marker (which has no offence anyway).
|
| 341 |
+
- {type: e1, owner: agent, position: [13, 12], stance: 0, spawn_point: 0}
|
| 342 |
+
# spawn_point 1 — SOUTH layout (bases at y=28). Per the
|
| 343 |
+
# CLAUDE.md spawn-filter rule, every base building is
|
| 344 |
+
# explicitly tagged for its group; otherwise the un-tagged
|
| 345 |
+
# actors would be filtered out and the agent would start
|
| 346 |
+
# bare.
|
| 347 |
+
- {type: fact, owner: agent, position: [15, 28], spawn_point: 1}
|
| 348 |
+
- {type: powr, owner: agent, position: [18, 28], spawn_point: 1}
|
| 349 |
+
- {type: fact, owner: agent, position: [85, 28], spawn_point: 1}
|
| 350 |
+
- {type: powr, owner: agent, position: [88, 28], spawn_point: 1}
|
| 351 |
+
- {type: e1, owner: agent, position: [13, 28], stance: 0, spawn_point: 1}
|
| 352 |
+
# Persistent unarmed enemy marker — not spawn_point-tagged
|
| 353 |
+
# (CLAUDE.md: enemy actors ignore the spawn_point filter).
|
| 354 |
+
- {type: fact, owner: enemy, position: [152, 52]}
|
| 355 |
+
win_condition:
|
| 356 |
+
all_of:
|
| 357 |
+
- has_building: fact
|
| 358 |
+
- any_of:
|
| 359 |
+
# NORTH match: both procs at y=12 band.
|
| 360 |
+
- all_of:
|
| 361 |
+
- building_in_region:
|
| 362 |
+
{x: 15, y: 12, radius: 8, type: proc, count: 1}
|
| 363 |
+
- building_in_region:
|
| 364 |
+
{x: 85, y: 12, radius: 8, type: proc, count: 1}
|
| 365 |
+
# SOUTH match: both procs at y=28 band.
|
| 366 |
+
- all_of:
|
| 367 |
+
- building_in_region:
|
| 368 |
+
{x: 15, y: 28, radius: 8, type: proc, count: 1}
|
| 369 |
+
- building_in_region:
|
| 370 |
+
{x: 85, y: 28, radius: 8, type: proc, count: 1}
|
| 371 |
+
- within_ticks: 1850
|
| 372 |
+
fail_condition:
|
| 373 |
+
any_of:
|
| 374 |
+
- after_ticks: 1851
|
| 375 |
+
- not: {has_building: fact}
|
| 376 |
+
max_turns: 23
|
|
@@ -0,0 +1,438 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""mfb-two-base-simultaneous — concurrent two-base ramp.
|
| 2 |
+
|
| 3 |
+
The bar: the intended CONCURRENT-PIPELINED policy (queue BOTH `proc`
|
| 4 |
+
orders on turn 1, then place each as soon as the engine reports it
|
| 5 |
+
ready) WINS on every level and every hard seed. Three failure modes
|
| 6 |
+
all LOSE on every level + seed:
|
| 7 |
+
• STALL — only `observe()` issued; nothing is built; after_ticks
|
| 8 |
+
fires.
|
| 9 |
+
• ONE-BASE-ONLY — build & place ONE proc only at the WEST base
|
| 10 |
+
(ignore the EAST base). The EAST `building_in_region:
|
| 11 |
+
proc` clause is never satisfied; after_ticks fires.
|
| 12 |
+
• SERIALIZE-WITH-IDLE-OBSERVE — naive build → observe-and-wait →
|
| 13 |
+
place → observe → build → observe → place. Adds ≥1
|
| 14 |
+
extra throttled turn between every productive command,
|
| 15 |
+
pushing the second `proc` placement to ≥ tick 1893 ≫
|
| 16 |
+
medium/hard within_ticks 1850 → after_ticks 1851 fires.
|
| 17 |
+
|
| 18 |
+
Validation is scripted (no model / network) — uses
|
| 19 |
+
`openra_bench.eval_core.run_level`.
|
| 20 |
+
"""
|
| 21 |
+
|
| 22 |
+
from __future__ import annotations
|
| 23 |
+
|
| 24 |
+
import types
|
| 25 |
+
from pathlib import Path
|
| 26 |
+
|
| 27 |
+
import pytest
|
| 28 |
+
|
| 29 |
+
pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
|
| 30 |
+
from openra_bench.scenarios import load_pack
|
| 31 |
+
from openra_bench.scenarios.loader import compile_level
|
| 32 |
+
from openra_bench.scenarios.win_conditions import WinContext, evaluate
|
| 33 |
+
|
| 34 |
+
PACKS = Path(__file__).parent.parent / "openra_bench" / "scenarios" / "packs"
|
| 35 |
+
PACK_PATH = PACKS / "mfb-two-base-simultaneous.yaml"
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
# ── unit-level predicate checks ──────────────────────────────────────
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def _ctx(own_buildings=(), tick=200):
|
| 42 |
+
sig = types.SimpleNamespace(
|
| 43 |
+
game_tick=tick,
|
| 44 |
+
units_killed=0,
|
| 45 |
+
units_lost=0,
|
| 46 |
+
own_buildings=list(own_buildings),
|
| 47 |
+
own_building_types={str(t).lower() for (t, _, _) in own_buildings},
|
| 48 |
+
enemies_seen_ids=set(),
|
| 49 |
+
enemy_buildings_seen_ids=set(),
|
| 50 |
+
)
|
| 51 |
+
return WinContext(signals=sig, render_state={"units_summary": []})
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def test_easy_predicates_require_proc_at_each_base():
|
| 55 |
+
c = compile_level(load_pack(PACK_PATH), "easy")
|
| 56 |
+
west_fact = ("fact", 15, 20)
|
| 57 |
+
east_fact = ("fact", 85, 20)
|
| 58 |
+
west_proc = ("proc", 15, 20)
|
| 59 |
+
east_proc = ("proc", 85, 20)
|
| 60 |
+
# WIN: facts alive + a proc at each base inside the budget.
|
| 61 |
+
assert evaluate(
|
| 62 |
+
c.win_condition,
|
| 63 |
+
_ctx([west_fact, east_fact, west_proc, east_proc], tick=500),
|
| 64 |
+
)
|
| 65 |
+
# FAIL clauses don't fire on the winning state.
|
| 66 |
+
assert not evaluate(
|
| 67 |
+
c.fail_condition,
|
| 68 |
+
_ctx([west_fact, east_fact, west_proc, east_proc], tick=500),
|
| 69 |
+
)
|
| 70 |
+
# FAIL: only WEST proc (east side never satisfied).
|
| 71 |
+
assert not evaluate(
|
| 72 |
+
c.win_condition,
|
| 73 |
+
_ctx([west_fact, east_fact, west_proc], tick=500),
|
| 74 |
+
)
|
| 75 |
+
# FAIL: only EAST proc.
|
| 76 |
+
assert not evaluate(
|
| 77 |
+
c.win_condition,
|
| 78 |
+
_ctx([west_fact, east_fact, east_proc], tick=500),
|
| 79 |
+
)
|
| 80 |
+
# FAIL: a third proc at the middle but neither edge — still fails.
|
| 81 |
+
middle_proc = ("proc", 50, 20)
|
| 82 |
+
assert not evaluate(
|
| 83 |
+
c.win_condition,
|
| 84 |
+
_ctx([west_fact, east_fact, middle_proc], tick=500),
|
| 85 |
+
)
|
| 86 |
+
# Past deadline ⇒ fail.
|
| 87 |
+
assert evaluate(
|
| 88 |
+
c.fail_condition,
|
| 89 |
+
_ctx([west_fact, east_fact, west_proc, east_proc], tick=2702),
|
| 90 |
+
)
|
| 91 |
+
# Lost every fact ⇒ fail (the not has_building:fact clause).
|
| 92 |
+
assert evaluate(c.fail_condition, _ctx([], tick=500))
|
| 93 |
+
# Deadline reachable inside max_turns (∼90 ticks/turn).
|
| 94 |
+
assert 2701 <= 93 + 90 * (c.max_turns - 1)
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
def test_medium_tight_clock_bites():
|
| 98 |
+
c = compile_level(load_pack(PACK_PATH), "medium")
|
| 99 |
+
west_fact = ("fact", 15, 20)
|
| 100 |
+
east_fact = ("fact", 85, 20)
|
| 101 |
+
west_proc = ("proc", 15, 20)
|
| 102 |
+
east_proc = ("proc", 85, 20)
|
| 103 |
+
# WIN before the tight deadline.
|
| 104 |
+
assert evaluate(
|
| 105 |
+
c.win_condition,
|
| 106 |
+
_ctx([west_fact, east_fact, west_proc, east_proc], tick=500),
|
| 107 |
+
)
|
| 108 |
+
# Past tick 1851 ⇒ fail.
|
| 109 |
+
assert evaluate(
|
| 110 |
+
c.fail_condition,
|
| 111 |
+
_ctx([west_fact, east_fact, west_proc, east_proc], tick=1852),
|
| 112 |
+
)
|
| 113 |
+
# Deadline reachable inside max_turns.
|
| 114 |
+
assert 1851 <= 93 + 90 * (c.max_turns - 1)
|
| 115 |
+
|
| 116 |
+
|
| 117 |
+
def test_hard_either_latitude_wins():
|
| 118 |
+
c = compile_level(load_pack(PACK_PATH), "hard")
|
| 119 |
+
# NORTH layout: facts at y=12.
|
| 120 |
+
n_w_fact = ("fact", 15, 12)
|
| 121 |
+
n_e_fact = ("fact", 85, 12)
|
| 122 |
+
n_w_proc = ("proc", 15, 12)
|
| 123 |
+
n_e_proc = ("proc", 85, 12)
|
| 124 |
+
assert evaluate(
|
| 125 |
+
c.win_condition,
|
| 126 |
+
_ctx([n_w_fact, n_e_fact, n_w_proc, n_e_proc], tick=500),
|
| 127 |
+
)
|
| 128 |
+
# SOUTH layout: facts at y=28.
|
| 129 |
+
s_w_fact = ("fact", 15, 28)
|
| 130 |
+
s_e_fact = ("fact", 85, 28)
|
| 131 |
+
s_w_proc = ("proc", 15, 28)
|
| 132 |
+
s_e_proc = ("proc", 85, 28)
|
| 133 |
+
assert evaluate(
|
| 134 |
+
c.win_condition,
|
| 135 |
+
_ctx([s_w_fact, s_e_fact, s_w_proc, s_e_proc], tick=500),
|
| 136 |
+
)
|
| 137 |
+
# MISMATCH: NORTH facts but a SOUTH-latitude proc on east — the
|
| 138 |
+
# NORTH clause needs east at y=12 (south proc at y=28 is too
|
| 139 |
+
# far); the SOUTH clause needs west AT y=28 (the NORTH proc is
|
| 140 |
+
# too far). Neither full-layout clause is satisfied.
|
| 141 |
+
assert not evaluate(
|
| 142 |
+
c.win_condition,
|
| 143 |
+
_ctx([n_w_fact, n_e_fact, n_w_proc, s_e_proc], tick=500),
|
| 144 |
+
)
|
| 145 |
+
# Past tick 1851 ⇒ fail.
|
| 146 |
+
assert evaluate(
|
| 147 |
+
c.fail_condition,
|
| 148 |
+
_ctx([n_w_fact, n_e_fact, n_w_proc, n_e_proc], tick=1852),
|
| 149 |
+
)
|
| 150 |
+
# Deadline reachable inside max_turns.
|
| 151 |
+
assert 1851 <= 93 + 90 * (c.max_turns - 1)
|
| 152 |
+
|
| 153 |
+
|
| 154 |
+
def test_hard_has_two_spawn_point_groups():
|
| 155 |
+
"""Hard-tier curation contract: ≥2 distinct agent spawn_point
|
| 156 |
+
groups so the seed round-robins the BASE LATITUDE (NORTH y=12
|
| 157 |
+
vs SOUTH y=28); the chosen target regions flip accordingly."""
|
| 158 |
+
c = compile_level(load_pack(PACK_PATH), "hard")
|
| 159 |
+
groups = {
|
| 160 |
+
(a.spawn_point if a.spawn_point is not None else 0)
|
| 161 |
+
for a in c.scenario.actors
|
| 162 |
+
if a.owner == "agent"
|
| 163 |
+
}
|
| 164 |
+
assert len(groups) >= 2, f"hard needs ≥2 spawn_point groups, got {groups}"
|
| 165 |
+
|
| 166 |
+
|
| 167 |
+
def test_pack_meta_fields_populated():
|
| 168 |
+
pack = load_pack(PACK_PATH)
|
| 169 |
+
assert pack.meta.id == "mfb-two-base-simultaneous"
|
| 170 |
+
assert pack.meta.capability == "reasoning"
|
| 171 |
+
anchors = pack.meta.benchmark_anchor
|
| 172 |
+
assert isinstance(anchors, list) and anchors, "benchmark_anchor required"
|
| 173 |
+
joined = " ".join(anchors).lower()
|
| 174 |
+
# Spec requires the SC2 / multi-plant / distributed anchors.
|
| 175 |
+
assert "sc2" in joined
|
| 176 |
+
assert "multi-plant" in joined or "manufacturing" in joined
|
| 177 |
+
assert "distributed" in joined
|
| 178 |
+
for lvl in ("easy", "medium", "hard"):
|
| 179 |
+
c = compile_level(pack, lvl)
|
| 180 |
+
assert c.map_supported
|
| 181 |
+
assert c.win_condition is not None and c.fail_condition is not None
|
| 182 |
+
# Tools must include build + place_building + observe (the
|
| 183 |
+
# whole task is build & place; deploy/move are unnecessary).
|
| 184 |
+
tools = set(c.scenario.tools or [])
|
| 185 |
+
for t in ("observe", "build", "place_building"):
|
| 186 |
+
assert t in tools, f"{lvl}: missing tool {t}"
|
| 187 |
+
|
| 188 |
+
|
| 189 |
+
def test_timeout_loss_reachable_every_level():
|
| 190 |
+
"""No draw degeneracy: the level's after_ticks fail must fit
|
| 191 |
+
inside max_turns on every level."""
|
| 192 |
+
pack = load_pack(PACK_PATH)
|
| 193 |
+
bars = {"easy": 2701, "medium": 1851, "hard": 1851}
|
| 194 |
+
for lvl, bar in bars.items():
|
| 195 |
+
c = compile_level(pack, lvl)
|
| 196 |
+
assert bar <= 93 + 90 * (c.max_turns - 1), lvl
|
| 197 |
+
|
| 198 |
+
|
| 199 |
+
def test_starting_cash_exactly_two_procs():
|
| 200 |
+
"""Cash sized 2 × proc cost (1400 each) — exactly two refineries;
|
| 201 |
+
a third would not fit so the bar cannot be brute-forced by
|
| 202 |
+
over-building."""
|
| 203 |
+
pack = load_pack(PACK_PATH)
|
| 204 |
+
assert pack.starting_cash == 2800
|
| 205 |
+
|
| 206 |
+
|
| 207 |
+
# ── engine-driven scripted policies ──────────────────────────────────
|
| 208 |
+
|
| 209 |
+
|
| 210 |
+
def _agent_own_buildings(rs):
|
| 211 |
+
"""Read agent-owned buildings from the render state.
|
| 212 |
+
|
| 213 |
+
The pack also includes an UNARMED enemy `fact` marker at (152,52)
|
| 214 |
+
to inhibit engine auto-`done`; the engine surfaces only
|
| 215 |
+
agent-owned buildings in `own_buildings`, so a simple type filter
|
| 216 |
+
is sufficient. (We still belt-and-suspenders strip any building
|
| 217 |
+
whose cell is the SE marker location.)"""
|
| 218 |
+
out = []
|
| 219 |
+
for b in (rs.get("own_buildings", []) or []):
|
| 220 |
+
cx = int(b.get("cell_x", 0))
|
| 221 |
+
cy = int(b.get("cell_y", 0))
|
| 222 |
+
# Skip the SE marker cell.
|
| 223 |
+
if cx == 152 and cy == 52:
|
| 224 |
+
continue
|
| 225 |
+
out.append(b)
|
| 226 |
+
return out
|
| 227 |
+
|
| 228 |
+
|
| 229 |
+
def _bases(rs):
|
| 230 |
+
"""Return [(label, cx, cy)] for WEST / EAST pre-placed facts,
|
| 231 |
+
sorted by cell_x ascending."""
|
| 232 |
+
facts = [b for b in _agent_own_buildings(rs) if str(b.get("type", "")).lower() == "fact"]
|
| 233 |
+
facts_sorted = sorted(facts, key=lambda f: int(f.get("cell_x", 0)))
|
| 234 |
+
out = []
|
| 235 |
+
for label, f in zip(("WEST", "EAST"), facts_sorted[:2]):
|
| 236 |
+
out.append((label, int(f["cell_x"]), int(f["cell_y"])))
|
| 237 |
+
return out
|
| 238 |
+
|
| 239 |
+
|
| 240 |
+
def _placed_proc_cells(rs):
|
| 241 |
+
return {
|
| 242 |
+
(int(b["cell_x"]), int(b["cell_y"]))
|
| 243 |
+
for b in _agent_own_buildings(rs)
|
| 244 |
+
if str(b.get("type", "")).lower() == "proc"
|
| 245 |
+
}
|
| 246 |
+
|
| 247 |
+
|
| 248 |
+
def _proc_pending(rs):
|
| 249 |
+
"""Count `proc` items currently in the agent's production list."""
|
| 250 |
+
prod = rs.get("production", []) or []
|
| 251 |
+
return sum(1 for item in prod if str(item).lower() == "proc")
|
| 252 |
+
|
| 253 |
+
|
| 254 |
+
def _intended_concurrent_policy(rs, Command):
|
| 255 |
+
"""Pipelined two-base ramp: keep the shared Building queue
|
| 256 |
+
saturated by queueing the NEXT `build proc` whenever any base
|
| 257 |
+
still needs a proc and the queue + already-placed procs together
|
| 258 |
+
won't cover them. Spam `place_building` at each base on every
|
| 259 |
+
turn (the engine just no-ops with PLACE BLOCKED until the build
|
| 260 |
+
actually completes — same idiom as
|
| 261 |
+
`mfb-third-base-against-clock`).
|
| 262 |
+
|
| 263 |
+
The placement is at (fact_x+5, fact_y) so the proc footprint
|
| 264 |
+
doesn't collide with the fact/powr footprints, and the cell sits
|
| 265 |
+
well inside the radius-8 region centred at (fact_x, fact_y).
|
| 266 |
+
"""
|
| 267 |
+
cmds: list = []
|
| 268 |
+
bases = _bases(rs)
|
| 269 |
+
if len(bases) < 2:
|
| 270 |
+
return [Command.observe()]
|
| 271 |
+
|
| 272 |
+
cash = int(rs.get("cash", 0) or 0)
|
| 273 |
+
placed_cells = _placed_proc_cells(rs)
|
| 274 |
+
in_queue = _proc_pending(rs)
|
| 275 |
+
|
| 276 |
+
# Which bases still need a proc inside their radius-8 region?
|
| 277 |
+
needed = []
|
| 278 |
+
for label, bx, by in bases:
|
| 279 |
+
if not any((px - bx) ** 2 + (py - by) ** 2 <= 64 for (px, py) in placed_cells):
|
| 280 |
+
needed.append((label, bx, by))
|
| 281 |
+
|
| 282 |
+
# Saturate the queue. Each `proc` costs 1400 ⇒ we can afford up
|
| 283 |
+
# to floor(cash/1400) more in the queue, capped by `needed`.
|
| 284 |
+
affordable = cash // 1400
|
| 285 |
+
short = max(0, len(needed) - in_queue)
|
| 286 |
+
to_queue = min(short, affordable)
|
| 287 |
+
for _ in range(to_queue):
|
| 288 |
+
cmds.append(Command.build("proc"))
|
| 289 |
+
|
| 290 |
+
# Spam place_building at each needed base.
|
| 291 |
+
for label, bx, by in needed:
|
| 292 |
+
cmds.append(Command.place_building("proc", bx + 5, by))
|
| 293 |
+
|
| 294 |
+
if not cmds:
|
| 295 |
+
return [Command.observe()]
|
| 296 |
+
return cmds
|
| 297 |
+
|
| 298 |
+
|
| 299 |
+
def _stall_policy(rs, Command):
|
| 300 |
+
return [Command.observe()]
|
| 301 |
+
|
| 302 |
+
|
| 303 |
+
def _one_base_only_policy(rs, Command):
|
| 304 |
+
"""Build and place a proc only at the WEST base — ignore the
|
| 305 |
+
EAST base entirely. The EAST region predicate is never
|
| 306 |
+
satisfied → after_ticks LOSS."""
|
| 307 |
+
bases = _bases(rs)
|
| 308 |
+
if not bases:
|
| 309 |
+
return [Command.observe()]
|
| 310 |
+
_, bx, by = bases[0]
|
| 311 |
+
placed_cells = _placed_proc_cells(rs)
|
| 312 |
+
cash = int(rs.get("cash", 0) or 0)
|
| 313 |
+
placed_west = any(
|
| 314 |
+
(px - bx) ** 2 + (py - by) ** 2 <= 64 for (px, py) in placed_cells
|
| 315 |
+
)
|
| 316 |
+
in_queue = _proc_pending(rs)
|
| 317 |
+
cmds: list = []
|
| 318 |
+
if not placed_west:
|
| 319 |
+
if in_queue == 0 and cash >= 1400:
|
| 320 |
+
cmds.append(Command.build("proc"))
|
| 321 |
+
cmds.append(Command.place_building("proc", bx + 5, by))
|
| 322 |
+
if not cmds:
|
| 323 |
+
return [Command.observe()]
|
| 324 |
+
return cmds
|
| 325 |
+
|
| 326 |
+
|
| 327 |
+
def _serialize_with_idle_observes_policy(rs, Command):
|
| 328 |
+
"""Naive sequential policy with idle observe gaps between every
|
| 329 |
+
productive command. The agent acts only on every OTHER turn (a
|
| 330 |
+
90-tick observe pause) and STRICTLY serializes the two bases:
|
| 331 |
+
build west → wait until placed → build east → wait until placed.
|
| 332 |
+
Total wall-clock at least ~tick 1900 ≫ medium/hard within_ticks
|
| 333 |
+
1850 → after_ticks 1851 fires → LOSS."""
|
| 334 |
+
|
| 335 |
+
tick = int(rs.get("game_tick", 0) or 0)
|
| 336 |
+
# Idle gap: act only when the turn index is even.
|
| 337 |
+
if (tick // 90) % 2 == 1:
|
| 338 |
+
return [Command.observe()]
|
| 339 |
+
|
| 340 |
+
bases = _bases(rs)
|
| 341 |
+
if len(bases) < 2:
|
| 342 |
+
return [Command.observe()]
|
| 343 |
+
|
| 344 |
+
cash = int(rs.get("cash", 0) or 0)
|
| 345 |
+
placed_cells = _placed_proc_cells(rs)
|
| 346 |
+
in_queue = _proc_pending(rs)
|
| 347 |
+
|
| 348 |
+
w_label, wx, wy = bases[0]
|
| 349 |
+
e_label, ex, ey = bases[1]
|
| 350 |
+
has_proc_west = any(
|
| 351 |
+
(px - wx) ** 2 + (py - wy) ** 2 <= 64 for (px, py) in placed_cells
|
| 352 |
+
)
|
| 353 |
+
has_proc_east = any(
|
| 354 |
+
(px - ex) ** 2 + (py - ey) ** 2 <= 64 for (px, py) in placed_cells
|
| 355 |
+
)
|
| 356 |
+
|
| 357 |
+
cmds: list = []
|
| 358 |
+
if not has_proc_west:
|
| 359 |
+
# Strictly one item in the queue at a time.
|
| 360 |
+
if in_queue == 0 and cash >= 1400:
|
| 361 |
+
cmds.append(Command.build("proc"))
|
| 362 |
+
cmds.append(Command.place_building("proc", wx + 5, wy))
|
| 363 |
+
elif not has_proc_east:
|
| 364 |
+
if in_queue == 0 and cash >= 1400:
|
| 365 |
+
cmds.append(Command.build("proc"))
|
| 366 |
+
cmds.append(Command.place_building("proc", ex + 5, ey))
|
| 367 |
+
|
| 368 |
+
if not cmds:
|
| 369 |
+
return [Command.observe()]
|
| 370 |
+
return cmds
|
| 371 |
+
|
| 372 |
+
|
| 373 |
+
@pytest.mark.parametrize("level", ["easy", "medium", "hard"])
|
| 374 |
+
def test_intended_concurrent_wins(level):
|
| 375 |
+
pytest.importorskip("openra_train")
|
| 376 |
+
from openra_bench.eval_core import run_level
|
| 377 |
+
|
| 378 |
+
c = compile_level(load_pack(PACK_PATH), level)
|
| 379 |
+
seeds = (1, 2, 3, 4) if level == "hard" else (1,)
|
| 380 |
+
for s in seeds:
|
| 381 |
+
res = run_level(c, _intended_concurrent_policy, seed=s)
|
| 382 |
+
assert res.outcome == "win", (
|
| 383 |
+
f"{level} seed={s}: intended concurrent should WIN, got "
|
| 384 |
+
f"{res.outcome} after {res.turns} turns; "
|
| 385 |
+
f"buildings={res.signals.own_buildings}"
|
| 386 |
+
)
|
| 387 |
+
|
| 388 |
+
|
| 389 |
+
@pytest.mark.parametrize("level", ["easy", "medium", "hard"])
|
| 390 |
+
def test_stall_loses(level):
|
| 391 |
+
pytest.importorskip("openra_train")
|
| 392 |
+
from openra_bench.eval_core import run_level
|
| 393 |
+
|
| 394 |
+
c = compile_level(load_pack(PACK_PATH), level)
|
| 395 |
+
seeds = (1, 2, 3, 4) if level == "hard" else (1,)
|
| 396 |
+
for s in seeds:
|
| 397 |
+
res = run_level(c, _stall_policy, seed=s)
|
| 398 |
+
assert res.outcome == "loss", (
|
| 399 |
+
f"{level} seed={s}: stall must LOSE, got {res.outcome} "
|
| 400 |
+
f"after {res.turns} turns; "
|
| 401 |
+
f"buildings={sorted(res.signals.own_building_types)}"
|
| 402 |
+
)
|
| 403 |
+
|
| 404 |
+
|
| 405 |
+
@pytest.mark.parametrize("level", ["easy", "medium", "hard"])
|
| 406 |
+
def test_one_base_only_loses(level):
|
| 407 |
+
pytest.importorskip("openra_train")
|
| 408 |
+
from openra_bench.eval_core import run_level
|
| 409 |
+
|
| 410 |
+
c = compile_level(load_pack(PACK_PATH), level)
|
| 411 |
+
seeds = (1, 2, 3, 4) if level == "hard" else (1,)
|
| 412 |
+
for s in seeds:
|
| 413 |
+
res = run_level(c, _one_base_only_policy, seed=s)
|
| 414 |
+
assert res.outcome == "loss", (
|
| 415 |
+
f"{level} seed={s}: one-base-only must LOSE (east region "
|
| 416 |
+
f"never covered), got {res.outcome} after {res.turns} "
|
| 417 |
+
f"turns; buildings={res.signals.own_buildings}"
|
| 418 |
+
)
|
| 419 |
+
|
| 420 |
+
|
| 421 |
+
@pytest.mark.parametrize("level", ["medium", "hard"])
|
| 422 |
+
def test_serialize_with_idle_observes_loses_on_tight_clocks(level):
|
| 423 |
+
"""The TIGHT-clock tiers (medium + hard) must distinguish
|
| 424 |
+
pipelined-concurrent from serialize-with-idle-observe; this
|
| 425 |
+
failure mode is allowed to WIN on easy (its generous clock is
|
| 426 |
+
deliberate — easy only filters out STALL)."""
|
| 427 |
+
pytest.importorskip("openra_train")
|
| 428 |
+
from openra_bench.eval_core import run_level
|
| 429 |
+
|
| 430 |
+
c = compile_level(load_pack(PACK_PATH), level)
|
| 431 |
+
seeds = (1, 2, 3, 4) if level == "hard" else (1,)
|
| 432 |
+
for s in seeds:
|
| 433 |
+
res = run_level(c, _serialize_with_idle_observes_policy, seed=s)
|
| 434 |
+
assert res.outcome == "loss", (
|
| 435 |
+
f"{level} seed={s}: serialize-with-idle-observes must "
|
| 436 |
+
f"LOSE on tight clock, got {res.outcome} after "
|
| 437 |
+
f"{res.turns} turns; buildings={res.signals.own_buildings}"
|
| 438 |
+
)
|