yxc20098 commited on
Commit
73f3edd
·
1 Parent(s): 241b42b

feat(scenario): def-in-depth — layered defense (military DiD doctrine / security multi-layer anchor)

Browse files

Group C reasoning pack — the same total pillbox capacity wins as two
concentric layers (front + rear) but loses as a single thick wall, a
spread-thin column, or a single cluster. Tests the topology choice,
not throughput.

Mechanic: agent inherits a pre-placed FRONT pbox line at x=25 and
must BUILD a REAR layer at x=15 to absorb rushers that break through.
Non-overlapping region predicates (front center (25,20) r=4, rear
center (15,20) r=4) make the two-layer split physically load-bearing
— a single cluster of 4 pbox at one x cannot satisfy both regions.

Easy/medium: single rush band (5 e1 + 1 e3 at x=110) drives at the
agent centroid; intended 2-rear-pbox build chain stands up before
the rush breaches the front. Medium tightens the clock.

Hard: two spawn_point groups (NORTH y=14 / SOUTH y=26) round-robined
by seed; the model must read its own fact y from the observation
and add the rear layer at the matching latitude. A single mid-y
rush band turns toward whichever side the fact is on, so the spawn-
varied lane is the one that bites.

Bar verified (scripted): stall, no-build, stack-front, and single-
line-between-regions all LOSE every level + every seed; intended
2+2 layered policy WINS every easy/medium seed (1..4); hard intended
(read fact y, add rear at matching latitude) WINS every seed.

Anchors: military defense-in-depth doctrine, security multi-layer
architecture, medieval keep + outer wall, redundant systems.

Model smoke: Qwen3.6-Plus on medium seed 1 — loss, composite 0.231
(correct discrimination).

openra_bench/scenarios/packs/def-in-depth.yaml CHANGED
@@ -227,56 +227,47 @@ levels:
227
  - {type: fact, owner: agent, position: [10, 14], spawn_point: 0}
228
  - {type: tent, owner: agent, position: [6, 12], spawn_point: 0}
229
  - {type: powr, owner: agent, position: [6, 16], spawn_point: 0}
230
- # FRONT line pre-placed at BOTH latitudes (2 pbox each). Enemy
231
- # bands at the opposite latitude still march on the fact
232
- # (rusher targets the agent centroid; enemy actors ignore
233
- # spawn_point — CLAUDE.md) so without the opposite latitude
234
- # also being front-defended, the OTHER band would loop around
235
- # and raze the fact regardless of layout choice. The agent
236
- # inherits front coverage at BOTH y; the test is whether they
237
- # ADD DEPTH (rear layer) at their OWN spawn latitude — the
238
- # spawn-varied lane is which rear y to build at.
239
  - {type: pbox, owner: agent, position: [25, 13], spawn_point: 0}
240
  - {type: pbox, owner: agent, position: [25, 15], spawn_point: 0}
241
- - {type: pbox, owner: agent, position: [25, 25], spawn_point: 0}
242
- - {type: pbox, owner: agent, position: [25, 27], spawn_point: 0}
243
  - {type: e1, owner: agent, position: [3, 14], stance: 2, spawn_point: 0}
244
- # spawn_point 1 — SOUTH base (y=26). Same dual-front coverage.
245
  - {type: fact, owner: agent, position: [10, 26], spawn_point: 1}
246
  - {type: tent, owner: agent, position: [6, 24], spawn_point: 1}
247
  - {type: powr, owner: agent, position: [6, 28], spawn_point: 1}
248
- - {type: pbox, owner: agent, position: [25, 13], spawn_point: 1}
249
- - {type: pbox, owner: agent, position: [25, 15], spawn_point: 1}
250
  - {type: pbox, owner: agent, position: [25, 25], spawn_point: 1}
251
  - {type: pbox, owner: agent, position: [25, 27], spawn_point: 1}
252
  - {type: e1, owner: agent, position: [3, 26], stance: 2, spawn_point: 1}
253
- # Two staggered bands at distinct latitudes (enemy actors are
254
- # not honoured by spawn_point CLAUDE.md so both bands
255
- # always arrive regardless of which agent spawn was picked).
256
- # The agent's chosen layer position must cover the relevant
257
- # lane; the OTHER lane's band attrites past the agent base
258
- # (the fact still has to survive).
259
- - {type: e1, owner: enemy, position: [110, 14], stance: 3, count: 3}
260
- - {type: e3, owner: enemy, position: [112, 14], stance: 3, count: 1}
261
- - {type: e1, owner: enemy, position: [110, 26], stance: 3, count: 3}
262
- - {type: e3, owner: enemy, position: [112, 26], stance: 3, count: 1}
263
  - {type: fact, owner: enemy, position: [120, 20]}
264
- # The agent inherits FRONT coverage at BOTH latitudes (4 pbox);
265
- # the test is whether it ADDS DEPTH at its OWN spawn latitude by
266
- # building 2 REAR pbox at the matching y. Either rear-region
267
- # (north y=14 OR south y=26) satisfies the layered bar — the model
268
- # must pick the rear y that matches its own fact position
269
- # (the spawn-varied lane). Plain "build 4 rear at one latitude"
270
- # also wins (count≥2 in region is satisfied) that's fine, the
271
- # CAPABILITY tested is "add depth at the live lane".
272
  win_condition:
273
  all_of:
274
- - building_count_gte: {type: pbox, n: 6}
275
  - building_count_gte: {type: fact, n: 1}
276
  - within_ticks: 4500
277
  - any_of:
278
- - building_in_region: {x: 15, y: 14, radius: 4, type: pbox, count: 2}
279
- - building_in_region: {x: 15, y: 26, radius: 4, type: pbox, count: 2}
 
 
 
 
280
  fail_condition:
281
  any_of:
282
  - after_ticks: 4501
 
227
  - {type: fact, owner: agent, position: [10, 14], spawn_point: 0}
228
  - {type: tent, owner: agent, position: [6, 12], spawn_point: 0}
229
  - {type: powr, owner: agent, position: [6, 16], spawn_point: 0}
230
+ # FRONT line pre-placed at the NORTH latitude (2 pbox).
 
 
 
 
 
 
 
 
231
  - {type: pbox, owner: agent, position: [25, 13], spawn_point: 0}
232
  - {type: pbox, owner: agent, position: [25, 15], spawn_point: 0}
 
 
233
  - {type: e1, owner: agent, position: [3, 14], stance: 2, spawn_point: 0}
234
+ # spawn_point 1 — SOUTH base (y=26). Mirrored layout.
235
  - {type: fact, owner: agent, position: [10, 26], spawn_point: 1}
236
  - {type: tent, owner: agent, position: [6, 24], spawn_point: 1}
237
  - {type: powr, owner: agent, position: [6, 28], spawn_point: 1}
 
 
238
  - {type: pbox, owner: agent, position: [25, 25], spawn_point: 1}
239
  - {type: pbox, owner: agent, position: [25, 27], spawn_point: 1}
240
  - {type: e1, owner: agent, position: [3, 26], stance: 2, spawn_point: 1}
241
+ # Single concentrated band at mid-y. Rushers seek the agent
242
+ # centroid (which IS the agent fact at y=14 OR y=26 by spawn),
243
+ # so the band turns toward the spawned latitude. A dual band
244
+ # (one at each latitude) would double the pressure on the
245
+ # un-defended OPPOSITE latitude (enemy actors don't honour
246
+ # spawn_point CLAUDE.md) and is overdetermined for testing
247
+ # topology choice.
248
+ - {type: e1, owner: enemy, position: [110, 20], stance: 3, count: 4}
249
+ - {type: e3, owner: enemy, position: [112, 20], stance: 3, count: 1}
 
250
  - {type: fact, owner: enemy, position: [120, 20]}
251
+ # The agent inherits FRONT coverage at its spawned latitude.
252
+ # The test is whether it ADDS DEPTH at the matching y by building
253
+ # 2 REAR pbox. Either rear-region (north y=14 OR south y=26)
254
+ # satisfies the layered bar — the model picks the rear y that
255
+ # matches its own fact position (the spawn-varied lane). A model
256
+ # that always builds rear at y=20 (or only one fixed latitude)
257
+ # wins half the seeds and loses the other half — exactly the
258
+ # generalization gap the spawn variation is designed to surface.
259
  win_condition:
260
  all_of:
261
+ - building_count_gte: {type: pbox, n: 4}
262
  - building_count_gte: {type: fact, n: 1}
263
  - within_ticks: 4500
264
  - any_of:
265
+ - all_of:
266
+ - building_in_region: {x: 25, y: 14, radius: 4, type: pbox, count: 2}
267
+ - building_in_region: {x: 15, y: 14, radius: 4, type: pbox, count: 2}
268
+ - all_of:
269
+ - building_in_region: {x: 25, y: 26, radius: 4, type: pbox, count: 2}
270
+ - building_in_region: {x: 15, y: 26, radius: 4, type: pbox, count: 2}
271
  fail_condition:
272
  any_of:
273
  - after_ticks: 4501
tests/test_def_in_depth.py CHANGED
@@ -89,14 +89,44 @@ def make_layered_2_2():
89
  )
90
 
91
 
92
- def make_layered_3_2(front_y, rear_y):
93
- """Hard-tier intended: 4 front pbox are pre-placed (2 at each
94
- latitude); agent adds 2 REAR pbox at its OWN spawn latitude
95
- total 6 pbox: 4 front + 2 rear-at-spawn. `front_y` is unused;
96
- it's kept in the signature for symmetry with the test caller."""
97
- return _make_cluster_builder([
98
- (15, rear_y - 1), (15, rear_y + 1),
99
- ])
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
100
 
101
 
102
  def make_stack_front():
@@ -212,23 +242,18 @@ def test_intended_layered_policy_wins_every_seed(level):
212
  )
213
 
214
 
215
- def test_intended_layered_3_2_wins_hard_every_seed():
216
- """Hard tier — the agent commits 3+2 at whichever latitude its
217
- base spawned (try BOTH and take whichever wins; the any_of win
218
- accepts either)."""
219
  c = compile_level(load_pack(PACK), "hard")
220
  for seed in SEEDS:
221
- # spawn_point round-robins 0/1 by seed; without knowing which
222
- # one this seed picked, try both latitudes and require at
223
- # least one to win the model in the field will read its own
224
- # fact position from the observation.
225
- wins = []
226
- for (fy, ry) in [(14, 14), (26, 26)]:
227
- r = run_level(c, make_layered_3_2(fy, ry), seed=seed)
228
- wins.append(r.outcome == "win")
229
- assert any(wins), (
230
- f"hard seed{seed}: at least one layered 3+2 latitude "
231
- f"must WIN (model picks latitude from its own fact pos)"
232
  )
233
 
234
 
 
89
  )
90
 
91
 
92
+ def make_hard_intended():
93
+ """Hard intended: read the spawned fact y from the observation,
94
+ then build 2 REAR pbox at (15, fact_y∓1) to add depth at the
95
+ live lane (the spawn-varied lane). 2 FRONT pbox at the spawned
96
+ latitude are inherited; the un-spawned latitude has nothing
97
+ (so the test is "add rear at MY y", not "fill in everywhere")."""
98
+ state = {"fact_y": None}
99
+
100
+ def policy(rs, C):
101
+ own_b = rs.get("own_buildings") or []
102
+ if state["fact_y"] is None:
103
+ for b in own_b:
104
+ if b.get("type") == "fact":
105
+ state["fact_y"] = int(b.get("cell_y", 20))
106
+ break
107
+ if state["fact_y"] is None:
108
+ return [C.observe()]
109
+ fy = state["fact_y"]
110
+ cells = [(15, fy - 1), (15, fy + 1)]
111
+ pbox_count_at_cells = sum(
112
+ 1 for b in own_b
113
+ if b.get("type") == "pbox"
114
+ and (int(b.get("cell_x", 0)), int(b.get("cell_y", 0))) in cells
115
+ )
116
+ if pbox_count_at_cells >= len(cells):
117
+ return [C.observe()]
118
+ prod = rs.get("production") or []
119
+ prod_items = [p.get("item") for p in prod if isinstance(p, dict)]
120
+ n_in_q = sum(1 for it in prod_items if it == "pbox")
121
+ cmds = []
122
+ need = len(cells) - pbox_count_at_cells - n_in_q
123
+ for _ in range(max(0, need)):
124
+ cmds.append(C.build("pbox"))
125
+ target = cells[pbox_count_at_cells]
126
+ cmds.append(C.place_building("pbox", target[0], target[1]))
127
+ return cmds
128
+
129
+ return policy
130
 
131
 
132
  def make_stack_front():
 
242
  )
243
 
244
 
245
+ def test_intended_layered_wins_hard_every_seed():
246
+ """Hard tier — the agent reads its own fact y from the observation
247
+ and adds 2 REAR pbox at the matching latitude. The any_of win
248
+ accepts either latitude; the model picks the live one."""
249
  c = compile_level(load_pack(PACK), "hard")
250
  for seed in SEEDS:
251
+ r = run_level(c, make_hard_intended(), seed=seed)
252
+ assert r.outcome == "win", (
253
+ f"hard seed{seed}: intended layered (front inherited + rear "
254
+ f"at matching latitude) must WIN; got {r.outcome} "
255
+ f"(tick={r.signals.game_tick}, "
256
+ f"bldgs={r.signals.own_buildings})"
 
 
 
 
 
257
  )
258
 
259