OpenRA-Bench / docs /ENGINE_FOLLOWUPS_TRIAGE.md
Xiaochuang Yuan
docs(engine-followups): refresh triage against post-merge OpenRA-Rust HEAD 8ca8989
b53d4ed
|
Raw
History Blame Contribute Delete
15.6 kB
# Engine Follow-ups โ€” Triage against `OpenRA-Rust/main` HEAD `8ca8989` (post-merge)
Cross-checks the seven findings in `docs/ENGINE_FOLLOWUPS.md` against
`OpenRA-Rust/main` HEAD `8ca8989` (the engine-feature-wave merge,
2026-05-23). The previous version of this doc (HEAD `feb8981`)
classified five findings as "unreproducible โ€” feature lives only on
`engine-feature-wave`"; that branch is now merged so each finding has
a real status against the current `main`.
## Executive summary
| # | Finding | Post-merge status | Action |
|---|---------|-------------------|--------|
| 1 | Tick-rate doc drift | DOC-ONLY (engine constant is 30/step) | **FIXED** โ€” `OpenRA-Bench/CLAUDE.md` updated |
| 2 | Ore density clamps at 12/cell | BROKEN (clamp still at 12) | DOC-FIXED โ€” `schema.py::ore_patches` docstring covers the recipe |
| 3 | Nuke AoE kills not credited to `kills_per_player` | **FIXED** (engine fix shipped this run) | Engine fix + 2 Rust tests landed |
| 4 | Explicit `harvest(unit_id, x, y)` drifts off target | BROKEN (harv re-binds to nearest patch after first deposit) | Failing `#[ignore]`d test pinned; **TODO engine fix** |
| 5 | ReturnFire stance breaks kite-and-pull | PARTIALLY FIXED (stance:1 fix landed); residual is balance/empirical | TODO empirical re-measure of `combat-kite-and-pull` |
| 6 | Arena generator's default 4-corner mpspawns | DOC-ONLY (auto-MCV varies per seed; engine doesn't offset pre-placed actors) | **FIXED** โ€” `ENGINE_FOLLOWUPS.md` rewording |
| 7 | Pre-placed `proc` overlapping `ore_patches:` | BROKEN (patch is seeded, but proc footprint blocks pathing โ€” silent sterilisation) | Failing `#[ignore]`d test pinned; **TODO validator** |
Wheel rebuilt cleanly: `Installed openra_train-0.1.0` on
`maturin develop --release` against HEAD `8ca8989`. `openra-sim` lib
+ integration tests are green except for known pre-existing
fixture/environment failures (see end of doc).
---
## Finding 1 โ€” Tick-rate discrepancy (doc drift)
**Status on `main`:** DOC-ONLY (engine constant is 30/step, not 90).
`openra-train/src/env.rs:33` declares `pub const
DEFAULT_TICKS_PER_STEP: u32 = 30;` and `Env::step` (env.rs:351-364)
advances exactly that many ticks per `step()`. The bench's
`RustEnvHandle.__init__` does not call `with_ticks_per_step` so every
`env.step` advances 30 ticks. Under `step_until_event` (interrupt
mode), the loop runs UP TO `max_ticks` (default 5 in bench code) and
breaks on the first signal โ€” per-turn advance is variable in
`[1, 5]`.
**Action taken:** `OpenRA-Bench/CLAUDE.md` updated in two spots
(defective-scenario list and engine-facts list). The "~90 ticks/turn"
phrase is gone; replaced with the actual `DEFAULT_TICKS_PER_STEP`
constant + a note to read `info["ticks_advanced"]` for interrupt-mode
runs.
---
## Finding 2 โ€” Ore density clamps at 12 cells
**Status on `main`:** BROKEN, but documented.
Clamp lives at `openra-sim/src/resource.rs:94-95`:
```rust
let per_cell = ((patch.amount.max(1) + passable_cells - 1) / passable_cells)
.clamp(1, 12) as u8;
```
Ceiling is 12 ore/cell; raising the cap is a real (though small)
balance change. The cheapest mitigation is the docstring (option
(c) in the spec) โ€” already applied to `schema.py`.
**Action taken:** `openra_bench/scenarios/schema.py::MapDef.ore_patches`
docstring now reproduces the clamp formula and the recommended ratio
`amount โ‰ˆ 12 ยท ฯ€ ยท radiusยฒ ยท 1.2`, plus the 50 cr/ore conversion. New
econ packs that use `ore_patches:` get the recipe inline.
**Optional follow-up (NOT in this commit):** raise the cap to 24/cell
in `resource.rs:95` and update the `seed_patch_fills_disk` density
assertion. Out of scope for triage; documented for whoever wants to
pick it up.
**Bench-side workarounds in place:** `econ-mine-and-grow.yaml`,
`econ-multi-patch-allocation.yaml`, `econ-second-base-race.yaml`,
`mcv-deploy-second-base.yaml` already inflate `amount` to fill the
cap; the new docstring formalises that recipe so future packs don't
re-discover the footgun.
---
## Finding 3 โ€” Nuke AoE kills not credited to `kills_per_player`
**Status on `main`:** **FIXED** in this triage pass.
Pre-fix `world.rs::detonate_nuke` (lines 2101-2141) removed dead
actors via `self.actors.remove(&id)` without ever incrementing
`kills_per_player`. The `_owner` parameter was even prefix-`_`'d to
silence the unused-variable warning, telegraphing the bug.
**Engine fix shipped:** `world.rs::detonate_nuke` now uses `owner`
and credits `kills_per_player` once per dead actor whose owner is
NOT the firing owner (mirrors the projectile-resolve scoping โ€”
friendly-fire kills do NOT credit, so an agent can't game
`units_killed_gte` by nuking its own units).
**Pinning:** `OpenRA-Rust/openra-sim/tests/test_nuke_credits_kills.rs`
โ€” two tests:
- `nuke_credits_each_enemy_kill_to_firing_owner` โ€” fire mslo at a
cluster of 5 enemy e1s, assert `kills_for_player(owner)` rises by
exactly 5.
- `nuke_does_not_credit_friendly_fire_kills` โ€” fire mslo at agent's
own e1 cluster, assert no credit.
**Existing test still green:** `test_superweapons.rs::mslo_nuke_kills_enemy_cluster`
runs unchanged (the new credit logic is additive).
**Bench-side cleanup queued (TODO, NOT this commit):**
- `OpenRA-Bench/openra_bench/scenarios/packs/spec-nuke-strike.yaml`
currently uses `enemy_buildings_destroyed_gte` with a `silo`-cluster
workaround (lines ~62-70 by triage finding #3). With the fix in
place, the hard-tier predicate can switch to `units_killed_gte: 5`
against an infantry cluster โ€” restoring the original "nuke clears
infantry cluster" semantics.
---
## Finding 4 โ€” Explicit `harvest(unit_id, x, y)` orders drift off target
**Status on `main`:** BROKEN.
`world.rs::order_harvest` (lines 1372-1402) is idempotent and stores
`last_harvest_cell: Some(target)` โ€” so the FIRST `FindingOre` cycle
honours the explicit target. But `tick_harvesters` (line 2261) calls
`terrain.find_nearest_resource(center, 15)` with `center =
last_harvest_cell.or(loc)`, and `last_harvest_cell` is overwritten
each successful harvest tick (line 2328) to the actually-harvested
cell. After the first deposit, the harv's "explicit target" has
already drifted to wherever it last mined; subsequent FindingOre
cycles search around that point and pick up whatever ore is closest,
without remembering which patch the original `harvest()` order
selected. With multiple patches inside a 15-cell window, the
explicit allocation evaporates within ~30 ticks.
**Pinning:** `openra-sim/tests/test_harvest_explicit_target.rs::explicit_harvest_target_stays_bound_to_far_patch`
โ€” `#[ignore]`d. Sets up a near patch (10,15) and far patch (40,15),
issues an explicit Harvest order targeting the FAR patch, runs 3000
ticks, asserts every harvested cell stays within Chebyshev distance
โ‰ค 4 of the far patch. Currently the harv would re-bind to whichever
patch is closer to its mining position. Flip to `#[test]` after the
fix lands.
**TODO engine fix:** add `bound_patch: Option<(x, y, radius)>` to
`Activity::Harvest`, set by `order_harvest` from the explicit target
(radius derived from the registered `OrePatchDef` containing the
cell, default 3). FindingOre's resource search restricts to cells
inside the patch disc when `bound_patch` is set. Cleared by a new
Harvest order to a different target or by `stop`. Multi-step change
(touches `Activity::Harvest` shape, `order_harvest`, `tick_harvesters`,
all `#[derive]`'d serialization tests) โ€” not a one-liner; pinned-test
+ TODO is the right form.
**Bench-side workarounds:** `econ-multi-patch-allocation.yaml:65-87,
212-216` pre-stages one harv beside each patch and relies on the
auto-route binding each to the nearest proc โ€” the workaround dodges
the explicit-allocation discipline by hand-placing harvs at the
target. Once the engine fix lands, the pack docstring and pre-staged
positions can be removed; `harvest()` becomes load-bearing again.
---
## Finding 5 โ€” ReturnFire-stance auto-fire breaks kite-and-pull
**Status on `main`:** PARTIALLY FIXED โ€” empirical re-measure needed.
The original finding's option (b) โ€” "Pure ReturnFire that fires only
after taking hits" โ€” landed in commit `9e999e2 fix(engine): clarify
stance โ€” stance:1 true return-fire-only`. Pinned by
`openra-sim/tests/test_stance_semantics.rs::test_stance_1_return_fire_only_against_passive_enemy`.
A stance:1 unit next to a passive (stance:0) enemy now holds fire
indefinitely.
But the **scenario** breaks for a different reason: once a stance:1
raider takes one hit from a 3tnk (hunt bot), it auto-fires forever
(the 60-tick window self-refreshes on each subsequent hit). So the
1v1 kite question becomes a balance question: does a passive
ReturnFire 2tnk out-trade an aggressive 3tnk by auto-fire alone? On
the OLD stance:1 (auto-engage on any in-range enemy) the answer was
yes โ€” the original finding's empirical claim. On the NEW stance:1
the answer depends on exact weapon DPS values which changed across
the wave merges; uncertain without a fresh empirical run.
**TODO empirical re-measure:** run the stand-still policy
(no agent orders) on `combat-kite-and-pull.yaml` against `main` HEAD
`8ca8989`. Suggested test:
`OpenRA-Rust/openra-sim/tests/test_kite_1v1.rs` โ€” 1 2tnk at (10,10)
stance:1 vs 1 3tnk at (40,10) stance:3 on a 60ร—30 arena, run 500
frames with no orders, assert 2tnk dies. If it passes, finding #5 is
genuinely fixed and the bench's 3-raider focus-fire workaround in
`combat-kite-and-pull.yaml:181-183` can be reverted to a true 1v1.
If it fails, the residual is balance: tune the 2tnk's `90mm` reload
delay or damage in `gamerules.rs`.
**Bench-side workaround:** `combat-kite-and-pull.yaml` uses a
3-raider stack to substitute focus-fire micro for distance-control
(documented inline). Workaround is correct given current uncertainty;
remove only after the empirical re-measure resolves.
---
## Finding 6 โ€” Arena generator's default 4-corner mpspawns rotate per-seed
**Status on `main`:** DOC-ONLY (engine code does not offset
pre-placed actor coords).
Re-reading the engine source: `build_scenario_actor`
(`openra-train/src/env.rs:2338+`) places each scenario actor at its
literal `sa.position` โ€” there is NO offset math anywhere. The
finding's claim "authored `position: [6, 5]` actually appears at
y=29-33 on some seeds" is not substantiated by the code. The
practical effect that DOES exist: `assign_spawn_points`
(`world.rs:4684+`) picks one mpspawn per seed for the auto-spawned
MCV, deterministic on `seed`. So when a pack relies on the
auto-MCV's corner being a specific one (not its position absolute),
that DOES vary per seed.
**Action taken:** `docs/ENGINE_FOLLOWUPS.md` finding #6 reworded to
remove the misleading "engine offsets pre-placed actors" claim.
**Bench-side workaround:** the `spawns: [[6, 5]]` declaration in
`perception-target-vs-fog.yaml:62-68, 157, 200, 252` (and any other
arena-generator pack that hand-places agent buildings AND relies on
a fixed auto-MCV corner) is correct and stays.
**Optional follow-up (NOT in this commit):** change
`openra_bench/mapgen.py::_default_spawns` default to a single centred
mpspawn so position invariants hold without the workaround. Touches
default behaviour for every existing arena-generator pack โ€” out of
scope for a doc-cleanup pass.
---
## Finding 7 โ€” Pre-placed `proc` inside an `ore_patches:` disc silently empties patch
**Status on `main`:** BROKEN (with refined understanding).
`env.rs::build_world_for_episode` lines 958-1029 seed ore patches
FIRST (line 966-974: `seed_ore_patch` for each declared patch), then
inject scenario actors. A `proc` placed inside a patch disc occupies
its footprint cells via `terrain.occupy_footprint`, marking them
ground-impassable WITHOUT clearing the resource layer. Result: ore
IS placed (the original finding's "silently zero ore" wording is
slightly off), but `find_path` cannot reach those cells from the
harvester. The harvester loops "FindingOre โ†’ can't path โ†’ fail"
indefinitely on the cells under the proc footprint.
**Pinning:** `openra-data/tests/test_proc_overlapping_patch_warns.rs`
โ€” `#[ignore]`d. Loads a scenario with `proc` at (40,10) and
`ore_patches: [{x:40, y:10, amount:5000, radius:3}]`, asserts the
(currently nonexistent) `validate_layout` returns a warning string
containing `proc_overlaps_patch`. Flip to `#[test]` after the
validator ships.
**TODO engine fix:** add `oramap::validate_layout(&MapDef) -> Vec<String>`
that returns one warning per overlapping `(building, patch)` pair.
Surface via `OpenRAEnv.last_warnings` at world-build. Cheap; the
optional second fix (skip ore-seeding on cells already occupied by a
building footprint, or re-order seed-after-actor-injection) is more
invasive โ€” leave for later.
**Bench-side workaround:** trivial (place the proc just outside the
patch radius). Already widely applied; no specific files to clean
up.
---
## Pre-existing test failures (NOT this triage pass)
Documented for clarity so they aren't conflated with finding-related
failures:
- `openra-sim` lib test `gamerules::tests::defaults_have_all_common_units`
โ€” MCV vs Vehicle kind classification regression, predates this
pass (logged in `OpenRA-Bench/ENGINE_AUDIT.md`).
- `openra-sim` integration tests `sync_hash_verify` (ร—2),
`debug_orders_and_hashes` โ€” sync-hash reference fixtures stale
after recent merges.
- `openra-sim` integration test `tank_attack_terminates_when_no_armament_loaded`
+ asset-loading data tests (`extract_*`, `find_*`,
`parse_infantry_yaml`, `e1_rifleman_typed_fields`,
`render_*`, etc.) โ€” all need the vendored OpenRA mod dir at
`OpenRA-Rust/vendor/OpenRA/mods/ra/` (not present in this
workspace; environmental, not code-related).
- `openra-data` `test_per_player_starting_cash`, etc. โ€” earlier these
failed with `MissingBaseMap` because the `rush-hour-arena.oramap`
fixture wasn't in `openra-data/tests/fixtures/`. Fixed in this pass
by copying the map from `OpenRA-Bench/data/maps/`. All four
per_player_starting_cash + 3 ore_patches data tests are green now.
## Summary of changes shipped this pass
**Engine (OpenRA-Rust):**
- `openra-sim/src/world.rs::detonate_nuke` โ€” credit nuke AoE kills to
firing owner (mirrors projectile-resolve path; friendly-fire is
excluded).
- `openra-sim/tests/test_nuke_credits_kills.rs` โ€” new (2 tests).
- `openra-sim/tests/test_harvest_explicit_target.rs` โ€” new
(`#[ignore]`d, pins finding #4).
- `openra-data/tests/test_proc_overlapping_patch_warns.rs` โ€” new
(`#[ignore]`d, pins finding #7).
- `openra-data/tests/fixtures/rush-hour-arena.oramap` โ€” copied in
from `OpenRA-Bench/data/maps/` so existing data tests run cleanly
in this workspace.
**Bench (OpenRA-Bench):**
- `CLAUDE.md` โ€” finding #1 doc fix in two places.
- `openra_bench/scenarios/schema.py::MapDef.ore_patches` docstring โ€”
finding #2 doc fix (clamp formula + recommended ratio).
- `docs/ENGINE_FOLLOWUPS_TRIAGE.md` โ€” this file (refreshed against
post-merge HEAD).
- `docs/ENGINE_FOLLOWUPS.md` โ€” finding #6 reworded to drop the
misleading "engine offsets pre-placed actors" claim.
**TODOs surfaced:**
- finding #4: implement `bound_patch` on `Activity::Harvest` (see test).
- finding #5: empirical re-measure on `combat-kite-and-pull` to
verify whether the stance:1 fix is sufficient.
- finding #7: implement `oramap::validate_layout` (see test).
- finding #2: optional cap raise from 12 โ†’ 24/cell.
- spec-nuke-strike.yaml: switch hard-tier predicate to
`units_killed_gte` now that finding #3 is fixed.