SCENARIO_UNIQUENESS_AUDIT.md

Scenario Uniqueness Audit (Phase 2)

Read-only audit of openra_bench/scenarios/packs/*.yaml. Scope: 212 YAML files (excluding TEMPLATE.yaml); 23 in-flight econ-* packs were deliberately skipped (other agents own them), leaving 189 packs probed.

The probe was a stall policy — Command.observe() only — run on the hard tier, seed=1, of every pack, through the same code path (openra_bench.eval_core.run_level) the bench evaluator uses. Each result is the engine-evaluated outcome (win / loss / draw), final game tick, and the static profile read straight from the YAML.

Headline numbers

• 181 / 189 packs (95.8%) — stall LOSSES (healthy: no-cheat bar holds).
• 3 packs — stall WINS (the no-cheat bar is broken).
• 3 packs — stall DRAWS (no real LOSS reachable from a stall play).
• 2 packs — Rust engine panic at reset(seed=1) (both adversarial-siege / adversarial-skirmish — already status: quarantine in-pack so they are NOT in the default eval set, but they remain on disk and panic when explicitly invoked).

Quarantined packs are still YAML-discoverable and were probed. They are flagged [Q] in the lists below.

---

Section 1 — Defect packs

A "defect" here means stall WINS (the lazy bar fell) or stall DRAWS (no real reachable LOSS — the timeout collapses to draw degeneracy because the fail predicate never trips). Both are explicit violations of the "no defect, no cheat" bar in CLAUDE.md.

1A. Stall WINS (3 — bar fell)

Pack	Outcome	Turn	Tick	Diagnosis
mid-economy-under-fire	WIN	11	993	Stall achieves the win predicate without any agent action. Hard win-clause is economy_value_gte:4000 AND harv≥2 AND units_lost_lte:2. With the current engine, the 3 starter harvesters auto-harvest (no harvest command needed — they begin in harvest mode), the perimeter 1tnks auto-fire and kill the lone raider 1tnk, and EV trips 4000 by tick ~993 with zero losses. The pack header says "Stall (only observe)… harvs never harvest → EV stays at 0 → LOSS." That is no longer true under the current engine — harvs in harvest mode work without re-issued orders. Fix: either remove the harvesters from the starting placement (require the model to issue Command.harvest) or replace the 1tnk raider with a stronger raider wave that out-attritions an idle defense ring.
combat-naval-shore-strike	WIN	3	198	Stall wins inside 3 turns. The hard tier places two destroyers in the water channel on stance:2 (Defend), and the entire shore garrison on stance:0 (HoldFire). The destroyers auto-fire on every garrison unit in range and clear all 5 enemies in <200 ticks; the agent never sends a command. Fix: set the destroyers to stance:0 so the agent must issue an attack_unit order; or place the shore garrison just outside the destroyer's auto-target range so the agent must manually fire. The whole capability (target an across-shore target with a ranged unit) collapses because the engine fires automatically.
def-with-ambush	(intended) WIN	24	1233	Not a defect — exempt by design. CLAUDE.md §Triage coverage: "1 pack (def-with-ambush) is exempt by design (positional-discipline scenario where do-nothing IS the intended policy)." The capability under test is hold the ambush position; the four stance:2 flanker tanks auto-engage the rusher band when it enters weapon range. Listed here only for completeness — no action required.

1B. Stall DRAWS (3 — no LOSS reachable)

A draw means the win predicate was never met AND the fail predicate never tripped, so the level resolves to draw (outcome score 0.5). This is a defect because it makes the lazy play indistinguishable from a partial / borderline play, and inflates the lazy score by treating it as half a win.

Pack	Outcome	Turn	Tick	Diagnosis
economy-time-box [Q]	DRAW	80	7203	Already quarantined in-file (reason: "redundant with economy-force-buildup"). The hard tier has no fail_condition at all (fail_condition: None), so a 12000-tick budget that never produces 6 units / 5 buildings simply times out as draw. This is one of the canonical CLAUDE.md defect classes ("no fail_condition, or only triggers on full force-wipe; a stall / preserve / partial outcome silently draws"). Since the pack is quarantined the lazy-bar drift is harmless to the default eval; if the pack is ever un-quarantined, add fail_condition: {after_ticks: 12001} plus {not: own_units_gte: 1}.
spec-spy-infiltrate	DRAW	3	273	Hard tier fail_condition: {after_ticks: 4001} — i.e. the only failure mode is the deadline. But the agent's only combat-relevant units are 2 spies (spy) and the scenario also places one defender e1. With stall, the spies are passive while the engine auto-dones at turn 3 (likely because the defender e1 falls to auto-fire or because all agent combat units are destroyed). The outcome lands as DRAW because the spy-only force is wiped before the within_ticks ever bites. Fix: add {not: own_units_gte: 1} (or unit_type_count_gte:{spy, 1}) to the fail_condition.any_of so a spy-wipe is a LOSS, not a draw. Same fix likely needed for easy / medium (uninspected; consider auditing).
def-bridge-chokepoint	(mixed: usually LOSS at tick 1623, occasionally DRAW in one fresh process)	18	1623	The first probe (subprocess #1) reported DRAW; three subsequent re-runs from a fresh process consistently reported LOSS (17 losses, fail-clause not has_building:fact trips when the procs/fact fall under the rifle assault). This is borderline — the pack is probably healthy, but the one-off DRAW observation suggests a possible determinism-edge case (race between the engine auto-done and the predicate evaluator). Recommend a 4-seed re-probe (1–4) over a clean Python process to confirm. If the LOSS is stable, no fix required; if the DRAW resurfaces, add {not: own_units_gte: 1} to the fail clause.

1C. Engine panics at reset (2)

Pack	Status	Error
adversarial-siege	quarantine (consolidated into adversarial-duel)	pathfinder.rs:175 index out of bounds: len 5120, index 5382 at env.reset(seed=1). The hard tier places actors outside the playable bounds (CLAUDE.md footgun #6) for the chosen spawn. Pack is excluded from default eval; no eval-time impact, but flag the file is still selectable via explicit --packs.
adversarial-skirmish	quarantine (consolidated into adversarial-duel)	Same panic at env.reset. Same situation as above.

Recommend either deleting these two YAMLs (the consolidation is the documented future state) or moving them to packs/_archived/ so discover_packs skips them (it already skips _* and TEMPLATE).

---

Section 2 — Duplicate / near-duplicate clusters

Methodology: each pack was fingerprinted by (capability, tools, actor_types_sorted, win_predicate_keys_sorted). Five strict (= all four fields equal) clusters surfaced; a second pass relaxed tools and actor_types to surface near-duplicates with the same capability + same win-key set, which surfaced a further ~10 clusters. Each cluster is judged on the per-pack real_world_meaning and the level_description of the hard tier — if these tell the same story, the cluster is genuinely redundant; if not, the predicate-key match is coincidental and the packs probe distinct skills.

2A. Strict duplicates (action required)

1. adversarial-siege + adversarial-skirmish — both already quarantined as "consolidated into adversarial-duel". Keep: adversarial-duel. Action: physical removal (delete or _archive) — they are the only two packs that crash the engine at load, so removing them eliminates the crash class entirely.

2. artofwar-decoy-sacrifice + artofwar-lure-the-tiger — both capability: reasoning, both use bot guard, identical hard-tier actor list [2tnk,e1,e3,fact,jeep], identical win-keys {units_in_region_gte, units_lost_lte, within_ticks}, both described as "main-force reaches an objective by diverting a leashed defender." Keep: artofwar-lure-the-tiger (the more doctrinally complete framing — leash mechanic; "the strong defender" is the load-bearing pull). Merge / delete: artofwar-decoy-sacrifice — its "spend a decoy unit" angle is covered by the units_lost_lte clause in the lure pack, which already permits a small attrition budget the model can spend on the bait sub-force.

3. scout-cycle-keep-info-fresh + scout-track-enemy-movement — both capability: perception, identical hard-tier actor list, both use the then-chained units_killed_gte win predicate, both probe "scout multiple times because the world changes between observations." Both use the scheduled_events: hook (cycle = reinforcement waves; track = enemy march legs). These are genuinely distinct skills — cycle is "re-observe a stale region for new arrivals", track is "follow a moving target across the map." Keep both, but verify the briefing of each clearly distinguishes the cause of staleness so the model is not asked to solve cycle-by-track-strategy or vice-versa. (No action required if the briefings are unambiguous, which they appear to be on inspection.)

4. build-defensive-skirt-corners + build-defensive-tower-line (strict 4-field match) AND the wider quartet build-defensive-tower-cluster + def-in-depth-vs-single (same capability + win-key set, near-identical actor list, all use rusher bot). These four packs all probe "where do you place a finite pillbox budget to survive an incoming rush?" with topology variants: skirt (one in each map-relative corner of the building), cluster (tight wrap), line (across the choke), depth (two thinner bands). Each topology is genuinely distinct in the optimal answer, so all four packs probe a real choice axis. Keep all four but consider promoting them into a single named "Defense Topology Suite" in the eval-cell catalog so the four cells score together (one "topology-IQ" metric) rather than as four unrelated reasoning packs. (No file action required; this is a catalog / metric grouping recommendation.)

2B. Near-duplicates by win-predicate keys (informational — same

predicate idiom is used across distinct scenarios; verify each is load-bearing)

These groups all share an identical (capability, win_predicate_keys) fingerprint. Each group below is not necessarily a duplicate, but flags that all members rely on the same predicate idiom and the scenarios should diverge on actor composition / pressure / bot to remain discriminative.

• (reasoning, then+within_ticks) — N=10: build-power-online-first, build-sequence-tech-cheapest, build-sequence-tech-fastest, build-sequence-tech-most-resilient (not in cluster — was filtered earlier), lh-econ-army-victory, lh-opening-to-defense-to-counter, lh-opening-to-tech-to-army, lh-progression-stage-locked, rob-objective-change-midway, tech-balanced-econ-then-tech, lh-build-army-coordinate-multifront-attack. These are all ordered-multi-phase scenarios. Distinct intents (build-order vs long-horizon vs objective-shift); the then predicate is the common engine machinery, not a duplication signal.

• (reasoning, building_count_gte+building_in_region+within_ticks) — N=7: build-sell-and-rebuild-elsewhere, def-in-depth, def-tower-line-vs-cluster, expansion-aggro-3-base-greedy, mcv-deploy-second-base, mcv-deploy-third-base, mfb-mirror-base-east-west. All probe "build / re-build N buildings in a target region." Genuinely distinct intents (expansion-greed, mirror-base symmetry, MCV-redeploy under pressure); keep all.

• (reasoning, building_count_gte+units_killed_gte+within_ticks) — N=6: adv-rps-counter-pick, build-tech-skip-decision, def-counter-battery, def-walls-vs-towers, def-while-building, lh-tech-rush-vs-army-rush. All "kill K enemies and have N buildings standing." Reads as honest differentiated tests.

• (action, own_units_gte+units_killed_gte+within_ticks) — N=5: combat-flanking-attack, combat-harass-aggro-commit, combat-kite-and-pull, combat-kite-jeep-vs-tank, combat-tanya-vs-rush. Five combat micro packs — kite, flank, harass. Each probes a different micro idiom; keep all but verify that the win bar (units_killed_gte) is tuned per pack so the intended micro is load-bearing, not just "auto-fire wins." Two are explicitly kite (combat-kite-and-pull, combat-kite-jeep-vs-tank) — possible duplicate: kite-and-pull uses generic 2tnk/3tnk vs jeep-vs-tank's unit-class asymmetry. Keep both only if combat-kite-jeep-vs-tank actually requires the jeep speed advantage that combat-kite-and-pull doesn't (read the briefings; they look distinct).

• (action, reach_region+units_lost_lte+within_ticks) — N=4: proc-no-attack-passive-only, proc-tool-use-multi-distractor, proc-tool-use-with-distractor, strict-toolban-fidelity-under-pressure. All procedural-compliance packs — "reach the goal without using a forbidden tool / without attacking." The proc-tool-use-* pair is a possible duplicate: "with-distractor" (one distractor tool) vs "multi-distractor" (multiple). Both are the same skill at different scales — recommend collapsing into a single pack with three tiers (no distractor / one / many) rather than two top-level packs.

• (perception, buildings_discovered_gte+units_lost_lte+within_ticks) — N=4: perception-target-vs-fog, scout-detect-enemy-tech, scout-discover-hidden-base, scout-multiple-fog-areas. All "discover K enemy buildings under fog." Each probes a distinct discrimination (target-vs-fog = ignore decoys; detect-enemy-tech = read the tech-tree; hidden-base = single hidden compound; multiple-fog-areas = K disjoint regions). Keep all.

2C. Overlapping prefixes worth a catalog review (not duplicates,

but the area is dense)

• combat-* (25 packs), def-* (18), scout-* (14), build-* (14), lh-* (13), proc-* (10), mfb-* (8), rob-* (8), tp-* (7), coord-* (7), mcv-* (6), spec-* (5), economy-* (5), tech-* (4), strategy-* (4), perception-* (4), artofwar-* (4), tempo-* (2), maint-* (2), mid-* (2), expansion-* (3), coordination-* (2), harass-* (1), risk-* (1), power-* (1), navigation-* (1), defense-* (1), longhorizon-* (1), custom-* (1), building-* (1), rush-* (1), reasoning-* (2), action-* (2), adv-* (2), adversarial-* (3), strict-* (3). The combat / def / scout / build axes carry ~70 packs combined; a follow-up audit explicitly checking pairwise overlap inside each prefix (briefing-level read) would surface ~5–10 more candidate collapses.

---

Section 3 — Capability coverage matrix

Cross-referenced against the phase × decision-type matrix sketched in PAPER_PLAN.md §12.2 and CLAUDE.md. Each cell lists the packs that primarily measure it (a pack may legitimately serve more than one cell — only the dominant assignment is listed; cross-cutting packs are flagged).

Opening

Decision	Packs	Density
MCV deploy site (where to plant)	mcv-deploy-and-build, mcv-deploy-defensible-site, mcv-deploy-near-resource, mcv-deploy-relocate-under-pressure, mcv-deploy-second-base, mcv-deploy-third-base	6 packs (dense; good)
Build-order commit	build-sequence-tech-cheapest, build-sequence-tech-fastest, build-sequence-tech-most-resilient, build-tech-skip-decision, tech-balanced-econ-then-tech, tech-aggro-all-in, tech-turtle-defensive-tech, tech-production-planning, build-power-online-first, power-budget-online, building-and-planning	11 packs (dense; good)
Defense-direction commit (which side to anticipate)	def-position-expected-direction, def-position-revealed-direction, def-multi-direction, def-surprise-flank-react, def-pre-position-mobile-reserve	5 packs (good)
Rush-defense	defense-rush-survive, rush-hour, build-defensive-tower-cluster, build-defensive-tower-line, build-defensive-skirt-corners, def-in-depth, def-in-depth-vs-single, def-walls-vs-towers, def-tower-line-vs-cluster, def-while-building	10 packs (dense)

Early-mid

Decision	Packs	Density
Harass / harass-preserve	harass-response-preserve, combat-harass-aggro-commit, combat-harass-balanced-hit-and-run, combat-skirmish-then-disengage, combat-bait-counter-attack, combat-kite-and-pull, combat-kite-jeep-vs-tank, combat-retreat-after-engagement, combat-suicide-charge-mission	9 packs (dense)
Exact-count perception	perception-count-the-threat, perception-count-the-threat-small-k, scout-count-defenders, scout-detect-incoming-army	4 packs (adequate)
Scout-direction commit	scout-detect-base-direction, scout-far-frontier, scout-frontier-reading (=perception-frontier-reading), scout-multiple-fog-areas, scout-and-report, scout-and-survive, scout-discover-hidden-base, scout-detect-enemy-tech, scout-map-reveal-percent-target, reasoning-frontier-commit	10 packs (dense; possible over-coverage)

Mid

Decision	Packs	Density
Live-economy defense	mid-economy-under-fire (DEFECT — stall wins), econ-harvester-defense-raid [skipped — econ-*], econ-protect-harvester-route [skipped]	~3 packs, 1 defective — the only non-econ exemplar is broken. Gap when the in-flight econ- land — verify they cover.*
Tech-switch on scout	mid-tech-switch-on-scout, lh-scout-react-counter, adv-rps-counter-pick	3 packs (adequate)
Second front	mfb-base-1-defend-base-2-build, mfb-supply-line-link-between-bases, mfb-mirror-base-east-west, mfb-two-base-simultaneous, mfb-third-base-against-clock, expansion-balanced-2-base-defended, expansion-aggro-3-base-greedy, coord-diversionary-attack	8 packs (dense)
Replan after loss	rob-unit-loss-recovery, rob-partial-base-loss-continue, lh-recovery-after-mid-game-loss, build-engineer-rebuild-after-loss, def-retreat-and-rebuild, build-sell-and-rebuild-elsewhere	6 packs (good)

Mid-late

Decision	Packs	Density
Concede vs hold	mid-concede-vs-hold	1 pack (THIN — single exemplar)
Isolate vs split	combat-divide-and-conquer, combat-pincer-coordination, combat-prevent-retreat, combat-formation-tank-wedge	4 packs (adequate)
Tempo double-window	tempo-double-window, tempo-strike-window, coordination-staggered-window, tp-survive-and-strike-at-window	4 packs (adequate)
Decoy / lure / feint	artofwar-decoy-sacrifice, artofwar-lure-the-tiger, artofwar-indirect-approach, artofwar-sequenced-citadel, combat-bait-counter-attack, coord-diversionary-attack, def-with-ambush	7 packs (good)
Multi-front	mfb-rotating-production-pressure, mfb-redundant-tech-buildings, mfb-tech-base-vs-economy-base, rob-multiple-simultaneous-pressures	4 packs (adequate)

Late

Decision	Packs	Density
Sustained multi-front	lh-build-army-coordinate-multifront-attack, mfb-rotating-production-pressure	2 packs (thin)
Base-trade race	None directly	0 packs (GAP)
Counter-strategy read	adv-rps-counter-pick, combat-attack-from-behind-fog, lh-tech-rush-vs-army-rush	3 packs (adequate)
Superweapon timing	spec-nuke-strike, spec-tanya-c4-strike, spec-engineer-capture, spec-spy-infiltrate (DEFECT — stall draws), spec-thief-steal-cash	5 packs, 1 defective (good once spec-spy is fixed)
Credit-only final phase	lh-credit-only-final-phase	1 pack (thin)

Cross-cutting

Capability	Packs	Density
Procedural compliance under pressure	proc-checklist-no-deviation, proc-conditional-branch-action, proc-instruction-following-edge-case, proc-no-attack-passive-only, proc-only-build-no-combat, proc-only-defend-no-attack, proc-ordered-action-strict, proc-strict-toolban-fidelity, proc-tool-use-multi-distractor, proc-tool-use-with-distractor, strict-production-bom, strict-sequence, strict-toolban-fidelity-under-pressure, tp-pressure-procedural	14 packs (very dense)
Long-horizon multi-phase	lh-* (13 packs), longhorizon-opening-to-assault, lh-100-turn-marathon-survival	14 packs (dense)
Coordination across squads	coord-converge-on-target, coord-cover-and-move, coord-diversionary-attack, coord-mutual-support, coord-relay-attack, coord-relay-vision-chain, coord-squad-handoff, coordination-ordered-rendezvous, coordination-staggered-window, action-multiunit-coordination, action-sequenced-execution, combat-pincer-coordination, combat-heli-flank	13 packs (dense)
Adversarial 1v1 (full macro)	adversarial-duel. The full 1v1 battleground lives in one_v_one.py (openra_bench/), not as a meta.capability: adversarial pack.	1 pack (the catalog says this is by design — full macro is the live ladder)

---

Section 4 — Capability gaps (uncovered or thin cells)

Cells worth new packs, in priority order:

1. Base-trade race (mid-late / late) — no current pack tests "your base falls while you race the enemy's; whoever finishes first wins." Sketch: hard tier places both players' bases halfway to killable, no defenders left; the agent has one army worth attacking with, the enemy is doing the same to the agent's base; win by destroying the enemy's fact before they destroy yours. Win building_count_gte:{type:fact,n:0,owner:enemy} paired with has_building:fact (own) and an aggressive within_ticks. Pack id: combat-base-trade-race.

2. Concede-vs-hold (mid-late) — only mid-concede-vs-hold currently. Sketch a second pack: mid-concede-vs-rebuild where a forward outpost is lost-cost (committing reinforcements throws good money after bad), and the right call is to abandon it and rebuild on a held line. Win condition: keep the rear-line fact alive AND have ≥4 tent/weap on the rear line at deadline; throwing reinforcements at the forward outpost fails the resource budget. Pack id: mid-concede-forward-rebuild-rear.

3. Live economy defense (mid) — the only non-econ pack (mid-economy-under-fire) is currently a stall-WIN defect. After fixing it, add a sibling pack mid-economy-rebuild-harvester-line where the agent must re-issue harvest commands to a freshly produced harvester after a raider kills the starter ones (puts the load-bearing capability on the harvest order, not on the passive auto-harvest).

4. Long-horizon credit-only final phase — lh-credit-only-final-phase is the only exemplar. Sketch sibling: lh-credit-only-bait-window — the agent must spend the last credits on a single decisive strike window rather than on attrition. Different decision (one shot vs accumulation).

5. Sustained multi-front (late) — thin (2 packs). Sketch: mfb-three-front-rotation — three simultaneously-attacked bases with the agent's army too small to defend all three at once; the decision is which two to hold and which one to let fall while the army cycles. Cross-pack with coord-relay-attack.

6. MCV deploy under timer + crossfire — current MCV packs cover site selection but not "deploy NOW or the MCV is destroyed by an incoming wave that hits in 4 turns." Sketch: mcv-deploy-emergency-relocation — the starting MCV stands in the path of an incoming squad; the agent must deploy + re-build OR move + re-deploy further west before the squad arrives.

7. Information freshness across modalities — scout-cycle-keep-info-fresh and scout-track-enemy-movement cover this for the structured channel, but the perception ablation grid (channel × fog) doesn't currently have an information-freshness pack that is explicitly easier to solve with the labelled image (the image channel is advantaged when the model can spot newly-spawned units inter-tick). Sketch: perception-freshness-image-advantage.

8. Single-pack meta.capability: adversarial — only adversarial-duel carries this tag. PAPER_PLAN.md §12.2 notes the imbalance. The full 1v1 lives in one_v_one.py (correct), but a second adversarial-tagged pack that tests "reactive opponent selects a counter from a small menu mid-game" (different from RPS pre-game commit) would put real teeth on the tag.

9. Engineer / Tanya / spy under fire (specialist packs) — only one pack each (spec-engineer-capture, spec-tanya-c4-strike, spec-spy-infiltrate, spec-thief-steal-cash, combat-tanya-vs-rush). Each specialist has one happy-path scenario. Sketch a "stealth + target priority" pack per specialist that requires the model to pick which of N enemy assets to hit with the one-shot specialist.

10. Naval / amphibious — combat-naval-shore-strike is the only naval pack (and it's currently a stall-WIN defect). Once fixed, add combat-naval-amphibious-landing (a transport ship lands infantry on a contested shore) and combat-naval-anti-air-vs-bomber pending the air-unit engine work. These are documented as out-of-scope in PAPER_PLAN.md §11.1 for the air variant but the naval ones are now feasible thanks to water_rect:.

---

Section 5 — Map mis-bindings (rush-hour-arena used where intent

calls for a custom map)

Of 189 probed packs, 183 use rush-hour-arena; 3 use a custom map (navigation-confined-hard-only, custom-map-no-enemy, strategy-dilemma/-gauntlet/-twobody); 1 uses a generator (combat-naval-shore-strike); 2 are quarantined-and-crashing.

Packs whose stated intent calls for a non-rush-hour geometry:

Pack	Stated intent	Current map	Recommendation
combat-heli-flank	Helicopter assault from a flank; the brief explicitly references aircraft flight over terrain.	rush-hour-arena (no terrain features to flank around)	Author / use a custom map with a forested ridge that channels ground units one way and aircraft another. (Engine air-unit support is out-of-scope per PAPER_PLAN §11.1 — pack is currently a ground-only proxy.)
def-bridge-chokepoint	"A water band cuts the map east-to-west with bridges."	rush-hour-arena + water_rect overlay	The water_rect overlay correctly synthesizes the bridge geometry on top of rush-hour, but the chokepoint-narrowing aspect ("attackers must pass through 3 narrow openings") works against an open-arena map. Consider a custom bridges-arena map authored once — the visual / minimap reading test is more honest when the bridges are real terrain rather than a YAML overlay on an unrelated map.
combat-naval-shore-strike	Naval ship in a water channel.	uses naval-arena generator (correct) — but the generator spec lives in the pack file rather than a named .oramap.	When a second naval pack is authored (per gap #10), promote naval-arena to a real data/maps/naval-arena.oramap so the bench has a canonical naval geometry.
def-with-ambush	"Concealed flanking defenders catch the band in an L-ambush down a lane toward the construction yard." Doctrinal answer requires real linear terrain.	rush-hour-arena (open)	The pack synthesizes the lane purely with actor placement (a single e1 fixing defender at x=15, flankers at x=40). Visually the geometry is invisible on the minimap — a custom corridor-arena map with a real lane would make the spatial decision legible from the image. (Capability is sound; this is a perception-channel polish issue.)
mfb-supply-line-link-between-bases	"Supply line corridor between two bases."	rush-hour-arena	Similar polish issue — a custom multi-base map with two separated valleys would make the supply-line decision visible on the minimap rather than implicit in the actor coordinates.
combat-hold-chokepoint	"Defend a narrow chokepoint corridor."	rush-hour-arena (no narrow corridor)	Same; either author a chokepoint-arena map or accept that the chokepoint is implied by enemy spawn geometry alone.
mcv-deploy-second-base, mcv-deploy-third-base	"Plant a second / third base at a defensible site."	rush-hour-arena (uniform; no defensible sites)	The map is symmetric and uniform — every cell is roughly as defensible as every other. The capability becomes "pick a cell that satisfies the building_in_region predicate," not "pick a defensible cell." Add a custom map with terrain features (cliff edges, narrow passes) so the defensibility geometry is real.
expansion-aggro-3-base-greedy, expansion-balanced-2-base-defended, expansion-turtle-1-base-fortified	Expansion decisions over distinct base sites.	rush-hour-arena (only ~3 sensible base sites given map size)	Authoring a wider arena with 5+ distinguishable expansion sites would put real teeth on the trilemma.

Summary: roughly 8–10 packs would benefit from a custom map, but all are currently functional (the no-cheat bar holds). The map mis-binding is a perception-channel polish issue, not a correctness issue. Priority: author 2 custom maps that unlock several packs each:

1. bridges-arena — would replace water_rect overlay on def-bridge-chokepoint and could host combat-naval-amphibious-landing.
2. chokepoint-arena — narrow corridor for combat-hold-chokepoint, def-with-ambush, mfb-supply-line-link-between-bases.

---

Section 6 — Recommended actions (prioritized)

P0 — Hard defects (gates the no-cheat headline number)

1. Fix mid-economy-under-fire — currently the only stall-WIN defect among non-exempt packs. Either:

   • (a) Remove the 3 starter harvesters; require the model to issue Command.harvest(...) on a fresh harvester (the load-bearing capability), OR
   • (b) Upgrade the raider to a force that out-attritions an idle defense ring (e.g. 3× 1tnk + 2× e3 over 60 turns), so a stall play loses harvesters and the harv,2 clause fails.

2. Fix combat-naval-shore-strike — set the destroyer stance to 0 (HoldFire) so the agent must issue an explicit attack_unit order. Verify the intended capability (manual cross-shore attack) still wins; stall now loses by deadline.

3. Fix spec-spy-infiltrate — add {not: own_units_gte: 1} (or {not: unit_type_count_gte: {type: spy, n: 1}}) to the fail_condition.any_of so a spy-wipe is a real LOSS, not a DRAW. Audit the easy / medium tiers for the same defect.

4. Delete or _archive/ adversarial-siege and adversarial-skirmish — both already quarantined; deletion removes the only two load-time-crashing packs.

5. Re-probe def-bridge-chokepoint over seeds 1–4 in clean processes — confirm the one-off DRAW observation was a transient (subsequent re-runs gave LOSS consistently); if the DRAW recurs, add {not: own_units_gte: 1} to its fail_condition.any_of.

P1 — Coverage gaps (new packs to author)

In priority order from §4: combat-base-trade-race, mid-concede-forward-rebuild-rear, mid-economy-rebuild-harvester-line, mfb-three-front-rotation, mcv-deploy-emergency-relocation, lh-credit-only-bait-window, perception-freshness-image-advantage, combat-naval-amphibious-landing, and per-specialist target-priority packs (spec-engineer-priority-targets, spec-tanya-priority-targets).

P2 — Duplicate consolidation

1. Merge / delete artofwar-decoy-sacrifice into artofwar-lure-the-tiger.
2. Merge proc-tool-use-with-distractor + proc-tool-use-multi-distractor into a single pack with two tiers.
3. Re-read combat-kite-and-pull vs combat-kite-jeep-vs-tank and either merge or sharpen the asymmetry-of-units distinction in the second pack's briefing.

P3 — Map / catalog polish

1. Author data/maps/bridges-arena.oramap and re-bind def-bridge-chokepoint + (future) combat-naval-amphibious-landing.
2. Author data/maps/chokepoint-arena.oramap and re-bind def-with-ambush, combat-hold-chokepoint, mfb-supply-line-link-between-bases.
3. Promote the four "defensive topology" packs (build-defensive-tower-cluster, build-defensive-tower-line, build-defensive-skirt-corners, def-in-depth-vs-single) into a single grouped eval-cell suite so they score one "topology-IQ" metric.

---

Appendix — Probe methodology

Each pack was probed by:

import openra_train  # builds the engine; required for env.reset
from openra_bench.scenarios.loader import compile_level, load_pack
from openra_bench.eval_core import run_level

pack = load_pack(pack_yaml_path)
c = compile_level(pack, "hard")
ep = run_level(c, lambda rs, Command: [Command.observe()], seed=1)
# ep.outcome ∈ {"win", "loss", "draw"}; ep.signals.game_tick = final tick.

The driver (/tmp/stall_probe_driver.py) invoked one subprocess per pack so a Rust engine panic in any one pack did not abort the run. 189 packs probed in 48 seconds wall-time (Apple M-series).

Hard tier seed=1 only. A full audit would extend to seeds 1–4 (the documented "hard seed" range in CLAUDE.md) and to easy/medium tiers, but seed-1 hard is the highest-pressure cell per pack — defects visible on any tier are visible here.

Static profile was extracted from the compiled CompiledLevel (authoritative — the engine sees the merged scenario, not the raw YAML).

Raw probe results JSON: /tmp/stall_probe_results.json (regeneratable by re-running /tmp/stall_probe_driver.py).