feat: add headful replay testing to Rust test runner
Browse filesMerge replay-test.ps1's headful mode into openwa-test as a new
`headful` subcommand. Runs with graphics+sound, checks for crashes,
panics, and [GAMEPLAY PASS/FAIL] markers. Supports filter by name
or direct .WAgame path, with configurable --timeout (default 150s).
replay-test.ps1 is now a thin wrapper like run-tests.ps1.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- .claude/skills/replay-test/SKILL.md +25 -27
- CLAUDE.md +19 -4
- crates/openwa-test-runner/CLAUDE.md +15 -3
- crates/openwa-test-runner/src/main.rs +340 -4
- replay-test.ps1 +5 -216
.claude/skills/replay-test/SKILL.md
CHANGED
|
@@ -1,13 +1,13 @@
|
|
| 1 |
---
|
| 2 |
name: replay-test
|
| 3 |
-
description: Run automated replay-based testing - builds, launches WA.exe with a replay file,
|
| 4 |
---
|
| 5 |
|
| 6 |
# Replay Test
|
| 7 |
|
| 8 |
Run the replay-based automated testing pipeline.
|
| 9 |
|
| 10 |
-
## Quick Start — Run All Tests
|
| 11 |
|
| 12 |
The fastest way to run all replay tests:
|
| 13 |
|
|
@@ -15,7 +15,7 @@ The fastest way to run all replay tests:
|
|
| 15 |
.\run-tests.ps1
|
| 16 |
```
|
| 17 |
|
| 18 |
-
This builds and runs the `openwa-test` binary
|
| 19 |
|
| 20 |
```
|
| 21 |
Building... done (2.1s)
|
|
@@ -35,48 +35,45 @@ Options:
|
|
| 35 |
|
| 36 |
## Modes
|
| 37 |
|
| 38 |
-
### Headless
|
| 39 |
|
| 40 |
Uses WA's built-in `/getlog` mode: no rendering, pure CPU simulation. Compares the output log byte-for-byte against an expected baseline.
|
| 41 |
|
| 42 |
```bash
|
| 43 |
-
# All tests:
|
| 44 |
.\run-tests.ps1
|
| 45 |
-
|
| 46 |
-
# Single replay via old script:
|
| 47 |
-
powershell -ExecutionPolicy Bypass -File replay-test.ps1 -Headless testdata/replays/longbow.WAgame
|
| 48 |
```
|
| 49 |
|
| 50 |
-
### Headful
|
| 51 |
|
| 52 |
-
Tests hooks, struct layouts, and game state with the full rendering pipeline active.
|
| 53 |
|
| 54 |
```bash
|
| 55 |
-
|
|
|
|
|
|
|
|
|
|
| 56 |
```
|
| 57 |
|
|
|
|
|
|
|
| 58 |
## Steps (for Claude)
|
| 59 |
|
| 60 |
1. Run the appropriate test command.
|
| 61 |
|
| 62 |
2. Read the output and present results:
|
| 63 |
|
| 64 |
-
**Headless
|
| 65 |
- PASS/FAIL summary is printed directly
|
| 66 |
- Per-run logs stored in `testdata/runs/<timestamp>/`
|
| 67 |
- FAIL shows first differing lines between expected and actual
|
| 68 |
|
| 69 |
-
**
|
| 70 |
-
- PASS
|
| 71 |
-
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
-
|
| 75 |
-
- Read `testdata/logs/validation_latest.log`:
|
| 76 |
-
- **Static checks** (`[STATIC PASS]`/`[STATIC FAIL]`): vtable addresses, struct offsets
|
| 77 |
-
- **Gameplay checks** (`[GAMEPLAY PASS]`/`[GAMEPLAY FAIL]`): game init, match start, match completion
|
| 78 |
-
- Quote any FAIL lines exactly
|
| 79 |
-
- Check `testdata/logs/openwa_latest.log` for `[PANIC]` errors
|
| 80 |
|
| 81 |
## Environment Variables
|
| 82 |
|
|
@@ -84,15 +81,15 @@ powershell -ExecutionPolicy Bypass -File replay-test.ps1
|
|
| 84 |
- `OPENWA_REPLAY_TEST=1` — (headful) Enable fast-forward mode (50x speed)
|
| 85 |
- `OPENWA_LOG_PATH=<path>` — Override OpenWA.log location (per-instance isolation)
|
| 86 |
|
| 87 |
-
All are set automatically by the test runner
|
| 88 |
|
| 89 |
## Key Paths
|
| 90 |
|
| 91 |
- Replay files + expected logs: `testdata/replays/*.WAgame` + `*_expected.log`
|
| 92 |
- Per-run output: `testdata/runs/<timestamp>/` (gitignored)
|
| 93 |
- Test runner: `crates/openwa-test-runner/` (`openwa-test` binary)
|
| 94 |
-
-
|
| 95 |
-
-
|
| 96 |
|
| 97 |
## Adding New Replay Tests
|
| 98 |
|
|
@@ -103,6 +100,7 @@ All are set automatically by the test runner / replay-test.ps1.
|
|
| 103 |
|
| 104 |
## Notes
|
| 105 |
|
| 106 |
-
- Headful replay: ~15-30s; headless: ~5s per test
|
|
|
|
| 107 |
- Concurrent headless tests use per-PID event names, log paths, and temp files (CreateFileA hook)
|
| 108 |
- Named event: `OpenWA_HooksReady_{pid}` — ensures hooks installed before WA main thread runs
|
|
|
|
| 1 |
---
|
| 2 |
name: replay-test
|
| 3 |
+
description: Run automated replay-based testing - builds, launches WA.exe with a replay file, and presents results. Supports headful (graphics + gameplay checks) and headless (pure simulation + log diff) modes via the openwa-test binary.
|
| 4 |
---
|
| 5 |
|
| 6 |
# Replay Test
|
| 7 |
|
| 8 |
Run the replay-based automated testing pipeline.
|
| 9 |
|
| 10 |
+
## Quick Start — Run All Tests (Headless)
|
| 11 |
|
| 12 |
The fastest way to run all replay tests:
|
| 13 |
|
|
|
|
| 15 |
.\run-tests.ps1
|
| 16 |
```
|
| 17 |
|
| 18 |
+
This builds and runs the `openwa-test` binary, which builds the DLL + launcher, discovers all `testdata/replays/*.WAgame` files with matching `*_expected.log`, and runs them concurrently (default 4 workers) in headless mode. Output:
|
| 19 |
|
| 20 |
```
|
| 21 |
Building... done (2.1s)
|
|
|
|
| 35 |
|
| 36 |
## Modes
|
| 37 |
|
| 38 |
+
### Headless (primary)
|
| 39 |
|
| 40 |
Uses WA's built-in `/getlog` mode: no rendering, pure CPU simulation. Compares the output log byte-for-byte against an expected baseline.
|
| 41 |
|
| 42 |
```bash
|
|
|
|
| 43 |
.\run-tests.ps1
|
| 44 |
+
.\run-tests.ps1 longbow
|
|
|
|
|
|
|
| 45 |
```
|
| 46 |
|
| 47 |
+
### Headful — graphics, sound, gameplay validation
|
| 48 |
|
| 49 |
+
Tests hooks, struct layouts, and game state with the full rendering pipeline active. Runs sequentially. The game window must be focused once for the replay to start advancing, after which it runs to completion on its own.
|
| 50 |
|
| 51 |
```bash
|
| 52 |
+
.\replay-test.ps1
|
| 53 |
+
.\replay-test.ps1 bots
|
| 54 |
+
.\replay-test.ps1 testdata/replays/bots.WAgame
|
| 55 |
+
.\replay-test.ps1 bots --timeout 600 # 10 minute timeout
|
| 56 |
```
|
| 57 |
|
| 58 |
+
Both scripts are thin wrappers around `openwa-test` / `openwa-test headful`.
|
| 59 |
+
|
| 60 |
## Steps (for Claude)
|
| 61 |
|
| 62 |
1. Run the appropriate test command.
|
| 63 |
|
| 64 |
2. Read the output and present results:
|
| 65 |
|
| 66 |
+
**Headless:**
|
| 67 |
- PASS/FAIL summary is printed directly
|
| 68 |
- Per-run logs stored in `testdata/runs/<timestamp>/`
|
| 69 |
- FAIL shows first differing lines between expected and actual
|
| 70 |
|
| 71 |
+
**Headful:**
|
| 72 |
+
- PASS/FAIL summary is printed directly
|
| 73 |
+
- Per-run logs stored in `testdata/runs/headful-<timestamp>/`
|
| 74 |
+
- Checks OpenWA.log for `[PANIC]` errors and `[GAMEPLAY PASS]`/`[GAMEPLAY FAIL]` markers
|
| 75 |
+
- PASS requires: no crash, no panics, no gameplay fails, at least one gameplay pass
|
| 76 |
+
- CRASH shows NTSTATUS name and ERRORLOG.TXT content
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
## Environment Variables
|
| 79 |
|
|
|
|
| 81 |
- `OPENWA_REPLAY_TEST=1` — (headful) Enable fast-forward mode (50x speed)
|
| 82 |
- `OPENWA_LOG_PATH=<path>` — Override OpenWA.log location (per-instance isolation)
|
| 83 |
|
| 84 |
+
All are set automatically by the test runner.
|
| 85 |
|
| 86 |
## Key Paths
|
| 87 |
|
| 88 |
- Replay files + expected logs: `testdata/replays/*.WAgame` + `*_expected.log`
|
| 89 |
- Per-run output: `testdata/runs/<timestamp>/` (gitignored)
|
| 90 |
- Test runner: `crates/openwa-test-runner/` (`openwa-test` binary)
|
| 91 |
+
- Headless script: `run-tests.ps1`
|
| 92 |
+
- Headful script: `replay-test.ps1`
|
| 93 |
|
| 94 |
## Adding New Replay Tests
|
| 95 |
|
|
|
|
| 100 |
|
| 101 |
## Notes
|
| 102 |
|
| 103 |
+
- Headful replay: ~15-30s per test; headless: ~5s per test
|
| 104 |
+
- Headful timeout defaults to 150s, configurable with `--timeout SECS`
|
| 105 |
- Concurrent headless tests use per-PID event names, log paths, and temp files (CreateFileA hook)
|
| 106 |
- Named event: `OpenWA_HooksReady_{pid}` — ensures hooks installed before WA main thread runs
|
CLAUDE.md
CHANGED
|
@@ -29,16 +29,31 @@ Unit tests: `cargo test`. These are standard Rust tests covering parsers and typ
|
|
| 29 |
|
| 30 |
WA.exe can deterministically replay recorded games (`.WAgame` files). Each replay test runs the game with the injected DLL and checks that the output matches a baseline log (`*_expected.log`) captured from unmodified WA.exe.
|
| 31 |
|
| 32 |
-
|
| 33 |
|
| 34 |
-
|
| 35 |
|
| 36 |
-
|
| 37 |
|
| 38 |
-
|
|
|
|
|
|
|
| 39 |
|
| 40 |
See `crates/openwa-test-runner/CLAUDE.md` for test isolation, crash detection, adding new tests, and env vars. Use `/desync-debug` skill after a test failure to diagnose.
|
| 41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
## Debug CLI
|
| 43 |
|
| 44 |
Use the `/debug-cli` skill for live memory inspection, struct inspection, frame-level control, and pointer chain traversal.
|
|
|
|
| 29 |
|
| 30 |
WA.exe can deterministically replay recorded games (`.WAgame` files). Each replay test runs the game with the injected DLL and checks that the output matches a baseline log (`*_expected.log`) captured from unmodified WA.exe.
|
| 31 |
|
| 32 |
+
Two ways to run replay tests:
|
| 33 |
|
| 34 |
+
### Headless (`.\run-tests.ps1`)
|
| 35 |
|
| 36 |
+
Pure CPU simulation, no rendering. Fast, runs in parallel (default 4 concurrent). Validates game logic — a log mismatch means the Rust code caused a desync. Spurious flakes from race conditions can occur; retry with `-j 1` to confirm.
|
| 37 |
|
| 38 |
+
### Headful (`.\replay-test.ps1` or `openwa-test headful`)
|
| 39 |
+
|
| 40 |
+
Runs with graphics and sound. The game window must be focused once to start. Needed to validate visual/audio hooks — code can pass headless but crash headful (or vice versa) since rendering and sound paths are only exercised headfully. Checks for crashes, panics, and `[GAMEPLAY PASS/FAIL]` markers. Timeout configurable with `--timeout SECS` (default 150s).
|
| 41 |
|
| 42 |
See `crates/openwa-test-runner/CLAUDE.md` for test isolation, crash detection, adding new tests, and env vars. Use `/desync-debug` skill after a test failure to diagnose.
|
| 43 |
|
| 44 |
+
### Use replay testing to validate assumptions and test theories
|
| 45 |
+
|
| 46 |
+
Implement a hypothesis, run tests, iterate. You can add temporary log statements and see their results by running replay tests.
|
| 47 |
+
|
| 48 |
+
### **IMPORTANT**: Replay tests can only test two things:
|
| 49 |
+
|
| 50 |
+
- The game simulation matches the original WA.exe, to the extent covered by the replay log and the replay's built-in checksums
|
| 51 |
+
- The game doesn't crash
|
| 52 |
+
|
| 53 |
+
You, as an agent, CANNOT see or hear the game, and therefore you can't verify that anything related to graphics or audio is correct. A passing headful replay test is NOT a guarantee that the rendering is correct. You need to involve the user to verify all graphics and audio related changes before committing or merging.
|
| 54 |
+
|
| 55 |
+
### **IMPORTANT:** The `*_expected.log` baselines are ground truth from unmodified WA.exe. They must NEVER be deleted or regenerated, unless explicitly requested by the user. If a test fails, the Rust code is wrong.
|
| 56 |
+
|
| 57 |
## Debug CLI
|
| 58 |
|
| 59 |
Use the `/debug-cli` skill for live memory inspection, struct inspection, frame-level control, and pointer chain traversal.
|
crates/openwa-test-runner/CLAUDE.md
CHANGED
|
@@ -1,6 +1,12 @@
|
|
| 1 |
# openwa-test-runner
|
| 2 |
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
## Test Isolation
|
| 6 |
|
|
@@ -18,6 +24,10 @@ All isolation mechanisms are active in headless mode only:
|
|
| 18 |
|
| 19 |
Tests that crash show `CRASH` (not `FAIL`) with NTSTATUS name and ERRORLOG.TXT content. ERRORLOG.TXT is redirected to the per-test run directory via `OPENWA_ERRORLOG_PATH`.
|
| 20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
## Adding New Replay Tests
|
| 22 |
|
| 23 |
1. Record a game in WA.exe (the replay `.WAgame` file is saved automatically)
|
|
@@ -28,11 +38,13 @@ Tests that crash show `CRASH` (not `FAIL`) with NTSTATUS name and ERRORLOG.TXT c
|
|
| 28 |
## Environment Variables
|
| 29 |
|
| 30 |
- `OPENWA_HEADLESS=1` — Headless mode: hooks MessageBoxA to auto-dismiss, launcher uses SW_HIDE, file isolation hook active, semaphore renamed per-PID
|
|
|
|
| 31 |
- `OPENWA_LOG_PATH=<path>` — Override OpenWA.log location (used for per-instance isolation)
|
| 32 |
- `OPENWA_ERRORLOG_PATH=<path>` — Redirect ERRORLOG.TXT to specified path (crash capture)
|
| 33 |
|
| 34 |
## Key Paths
|
| 35 |
|
| 36 |
- Replay files + expected logs: `testdata/replays/*.WAgame` + `*_expected.log`
|
| 37 |
-
- Per-run output: `testdata/runs/<timestamp>/` (
|
| 38 |
-
-
|
|
|
|
|
|
| 1 |
# openwa-test-runner
|
| 2 |
|
| 3 |
+
Replay test runner (`openwa-test` binary). Supports headless (concurrent, log-diff) and headful (sequential, gameplay checks) modes.
|
| 4 |
+
|
| 5 |
+
## Subcommands
|
| 6 |
+
|
| 7 |
+
- `openwa-test [filter] [-j N] [--no-build] [--wa-path PATH]` — Headless: discovers `testdata/replays/*.WAgame` with `*_expected.log`, runs concurrently via `/getlog`, compares output byte-for-byte.
|
| 8 |
+
- `openwa-test headful [filter|replay.WAgame] [--no-build] [--wa-path PATH] [--timeout SECS]` — Headful: runs with graphics+sound, checks for crashes/panics/gameplay markers. Default timeout 150s.
|
| 9 |
+
- `openwa-test trace-desync <replay.WAgame> [--no-build] [--wa-path PATH]` — Compares per-frame checksums between baseline and hooked runs.
|
| 10 |
|
| 11 |
## Test Isolation
|
| 12 |
|
|
|
|
| 24 |
|
| 25 |
Tests that crash show `CRASH` (not `FAIL`) with NTSTATUS name and ERRORLOG.TXT content. ERRORLOG.TXT is redirected to the per-test run directory via `OPENWA_ERRORLOG_PATH`.
|
| 26 |
|
| 27 |
+
## Headful Pass Criteria
|
| 28 |
+
|
| 29 |
+
A headful test passes when: no crash, no `[PANIC]` in OpenWA.log, no `[GAMEPLAY FAIL]`, and at least one `[GAMEPLAY PASS]` marker present.
|
| 30 |
+
|
| 31 |
## Adding New Replay Tests
|
| 32 |
|
| 33 |
1. Record a game in WA.exe (the replay `.WAgame` file is saved automatically)
|
|
|
|
| 38 |
## Environment Variables
|
| 39 |
|
| 40 |
- `OPENWA_HEADLESS=1` — Headless mode: hooks MessageBoxA to auto-dismiss, launcher uses SW_HIDE, file isolation hook active, semaphore renamed per-PID
|
| 41 |
+
- `OPENWA_REPLAY_TEST=1` — Headful mode: enables fast-forward replay (50x speed)
|
| 42 |
- `OPENWA_LOG_PATH=<path>` — Override OpenWA.log location (used for per-instance isolation)
|
| 43 |
- `OPENWA_ERRORLOG_PATH=<path>` — Redirect ERRORLOG.TXT to specified path (crash capture)
|
| 44 |
|
| 45 |
## Key Paths
|
| 46 |
|
| 47 |
- Replay files + expected logs: `testdata/replays/*.WAgame` + `*_expected.log`
|
| 48 |
+
- Per-run output: `testdata/runs/<timestamp>/` (headless), `testdata/runs/headful-<timestamp>/` (headful)
|
| 49 |
+
- Headless script: `run-tests.ps1`
|
| 50 |
+
- Headful script: `replay-test.ps1`
|
crates/openwa-test-runner/src/main.rs
CHANGED
|
@@ -667,6 +667,334 @@ fn parse_errorlog_exception(content: Option<&str>) -> &'static str {
|
|
| 667 |
"crash (SEH handled)"
|
| 668 |
}
|
| 669 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 670 |
// ─── Trace-desync subcommand ─────────────────────────────────────────────
|
| 671 |
|
| 672 |
struct TraceDesyncArgs {
|
|
@@ -995,10 +1323,18 @@ fn report_startup_check_failures(results: &[TestResult], run_dir: &Path) {
|
|
| 995 |
fn main() {
|
| 996 |
// Check for subcommands before normal arg parsing
|
| 997 |
let argv: Vec<String> = env::args().skip(1).collect();
|
| 998 |
-
|
| 999 |
-
|
| 1000 |
-
|
| 1001 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1002 |
}
|
| 1003 |
|
| 1004 |
let args = parse_args();
|
|
|
|
| 667 |
"crash (SEH handled)"
|
| 668 |
}
|
| 669 |
|
| 670 |
+
// ─── Headful subcommand ────────────────────────────────────────────────────
|
| 671 |
+
|
| 672 |
+
const DEFAULT_HEADFUL_TIMEOUT_SECS: u64 = 150;
|
| 673 |
+
|
| 674 |
+
struct HeadfulArgs {
|
| 675 |
+
filter_or_replay: Option<String>,
|
| 676 |
+
no_build: bool,
|
| 677 |
+
wa_path: Option<PathBuf>,
|
| 678 |
+
timeout_secs: u64,
|
| 679 |
+
}
|
| 680 |
+
|
| 681 |
+
fn parse_headful_args(argv: &[String]) -> HeadfulArgs {
|
| 682 |
+
let mut args = HeadfulArgs {
|
| 683 |
+
filter_or_replay: None,
|
| 684 |
+
no_build: false,
|
| 685 |
+
wa_path: None,
|
| 686 |
+
timeout_secs: DEFAULT_HEADFUL_TIMEOUT_SECS,
|
| 687 |
+
};
|
| 688 |
+
let mut i = 0;
|
| 689 |
+
while i < argv.len() {
|
| 690 |
+
match argv[i].as_str() {
|
| 691 |
+
"--no-build" => args.no_build = true,
|
| 692 |
+
"--wa-path" => {
|
| 693 |
+
i += 1;
|
| 694 |
+
if i < argv.len() {
|
| 695 |
+
args.wa_path = Some(PathBuf::from(&argv[i]));
|
| 696 |
+
}
|
| 697 |
+
}
|
| 698 |
+
"--timeout" | "-t" => {
|
| 699 |
+
i += 1;
|
| 700 |
+
if i < argv.len() {
|
| 701 |
+
args.timeout_secs = argv[i]
|
| 702 |
+
.parse()
|
| 703 |
+
.unwrap_or(DEFAULT_HEADFUL_TIMEOUT_SECS);
|
| 704 |
+
}
|
| 705 |
+
}
|
| 706 |
+
s if !s.starts_with('-') && args.filter_or_replay.is_none() => {
|
| 707 |
+
args.filter_or_replay = Some(s.to_string());
|
| 708 |
+
}
|
| 709 |
+
other => {
|
| 710 |
+
eprintln!("Unknown argument: {other}");
|
| 711 |
+
eprintln!(
|
| 712 |
+
"Usage: openwa-test headful [filter|replay.WAgame] [--no-build] [--wa-path PATH] [--timeout SECS]"
|
| 713 |
+
);
|
| 714 |
+
std::process::exit(1);
|
| 715 |
+
}
|
| 716 |
+
}
|
| 717 |
+
i += 1;
|
| 718 |
+
}
|
| 719 |
+
args
|
| 720 |
+
}
|
| 721 |
+
|
| 722 |
+
/// Discover replays for headful testing. Unlike headless, doesn't require _expected.log.
|
| 723 |
+
/// If `filter_or_replay` ends with .WAgame and exists, treat it as a direct path.
|
| 724 |
+
fn discover_headful_tests(filter_or_replay: Option<&str>) -> Vec<TestCase> {
|
| 725 |
+
// Direct .WAgame path?
|
| 726 |
+
if let Some(arg) = filter_or_replay {
|
| 727 |
+
let path = Path::new(arg);
|
| 728 |
+
if arg.ends_with(".WAgame") && path.exists() {
|
| 729 |
+
let abs = strip_unc(fs::canonicalize(path).unwrap_or(path.to_path_buf()));
|
| 730 |
+
let stem = abs
|
| 731 |
+
.file_stem()
|
| 732 |
+
.and_then(|s| s.to_str())
|
| 733 |
+
.unwrap_or("unknown")
|
| 734 |
+
.to_string();
|
| 735 |
+
return vec![TestCase {
|
| 736 |
+
name: stem,
|
| 737 |
+
replay_path: abs.clone(),
|
| 738 |
+
expected_log: abs.with_extension("expected.log"),
|
| 739 |
+
output_log: abs.with_extension("log"),
|
| 740 |
+
}];
|
| 741 |
+
}
|
| 742 |
+
}
|
| 743 |
+
|
| 744 |
+
// Scan replays dir with optional filter
|
| 745 |
+
let replays_dir = Path::new(REPLAYS_DIR);
|
| 746 |
+
let entries = match fs::read_dir(replays_dir) {
|
| 747 |
+
Ok(e) => e,
|
| 748 |
+
Err(e) => {
|
| 749 |
+
eprintln!("Cannot read {REPLAYS_DIR}: {e}");
|
| 750 |
+
return Vec::new();
|
| 751 |
+
}
|
| 752 |
+
};
|
| 753 |
+
|
| 754 |
+
let mut tests = Vec::new();
|
| 755 |
+
for entry in entries.flatten() {
|
| 756 |
+
let path = entry.path();
|
| 757 |
+
if path.extension().and_then(|e| e.to_str()) != Some("WAgame") {
|
| 758 |
+
continue;
|
| 759 |
+
}
|
| 760 |
+
let stem = match path.file_stem().and_then(|s| s.to_str()) {
|
| 761 |
+
Some(s) => s.to_string(),
|
| 762 |
+
None => continue,
|
| 763 |
+
};
|
| 764 |
+
if let Some(filter) = filter_or_replay {
|
| 765 |
+
if !stem.contains(filter) {
|
| 766 |
+
continue;
|
| 767 |
+
}
|
| 768 |
+
}
|
| 769 |
+
// No _expected.log requirement for headful
|
| 770 |
+
let abs = strip_unc(fs::canonicalize(&path).unwrap_or(path.clone()));
|
| 771 |
+
tests.push(TestCase {
|
| 772 |
+
name: stem,
|
| 773 |
+
replay_path: abs.clone(),
|
| 774 |
+
expected_log: abs.with_extension("expected.log"),
|
| 775 |
+
output_log: abs.with_extension("log"),
|
| 776 |
+
});
|
| 777 |
+
}
|
| 778 |
+
tests.sort_by(|a, b| a.name.cmp(&b.name));
|
| 779 |
+
tests
|
| 780 |
+
}
|
| 781 |
+
|
| 782 |
+
/// Analyze an OpenWA.log for panics and gameplay check markers.
|
| 783 |
+
struct HeadfulLogAnalysis {
|
| 784 |
+
panics: Vec<String>,
|
| 785 |
+
gameplay_passes: Vec<String>,
|
| 786 |
+
gameplay_fails: Vec<String>,
|
| 787 |
+
}
|
| 788 |
+
|
| 789 |
+
fn analyze_headful_log(log_path: &Path) -> HeadfulLogAnalysis {
|
| 790 |
+
let content = fs::read_to_string(log_path).unwrap_or_default();
|
| 791 |
+
let mut analysis = HeadfulLogAnalysis {
|
| 792 |
+
panics: Vec::new(),
|
| 793 |
+
gameplay_passes: Vec::new(),
|
| 794 |
+
gameplay_fails: Vec::new(),
|
| 795 |
+
};
|
| 796 |
+
for line in content.lines() {
|
| 797 |
+
if line.contains("[PANIC]") {
|
| 798 |
+
analysis.panics.push(line.to_string());
|
| 799 |
+
} else if line.contains("[GAMEPLAY PASS]") {
|
| 800 |
+
analysis.gameplay_passes.push(line.to_string());
|
| 801 |
+
} else if line.contains("[GAMEPLAY FAIL]") {
|
| 802 |
+
analysis.gameplay_fails.push(line.to_string());
|
| 803 |
+
}
|
| 804 |
+
}
|
| 805 |
+
analysis
|
| 806 |
+
}
|
| 807 |
+
|
| 808 |
+
/// Run a single headful replay test.
|
| 809 |
+
fn run_headful_test(
|
| 810 |
+
test: &TestCase,
|
| 811 |
+
launcher: &Path,
|
| 812 |
+
wa_exe: &Path,
|
| 813 |
+
run_dir: &Path,
|
| 814 |
+
timeout_secs: u64,
|
| 815 |
+
) -> TestResult {
|
| 816 |
+
let start = Instant::now();
|
| 817 |
+
|
| 818 |
+
let openwa_log = run_dir.join(format!("{}.openwa.log", test.name));
|
| 819 |
+
let errorlog_path = run_dir.join(format!("{}.errorlog.txt", test.name));
|
| 820 |
+
|
| 821 |
+
let result = Command::new(launcher)
|
| 822 |
+
.arg("--minimized")
|
| 823 |
+
.arg(wa_exe)
|
| 824 |
+
.arg(&test.replay_path)
|
| 825 |
+
.env("OPENWA_REPLAY_TEST", "1")
|
| 826 |
+
.env("OPENWA_LOG_PATH", &openwa_log)
|
| 827 |
+
.env("OPENWA_ERRORLOG_PATH", &errorlog_path)
|
| 828 |
+
.stdout(std::process::Stdio::null())
|
| 829 |
+
.stderr(std::process::Stdio::null())
|
| 830 |
+
.status();
|
| 831 |
+
|
| 832 |
+
let duration = start.elapsed();
|
| 833 |
+
|
| 834 |
+
match result {
|
| 835 |
+
Err(e) => TestResult {
|
| 836 |
+
name: test.name.clone(),
|
| 837 |
+
passed: false,
|
| 838 |
+
duration,
|
| 839 |
+
diff_lines: Vec::new(),
|
| 840 |
+
error: Some(format!("Failed to launch: {e}")),
|
| 841 |
+
crashed: None,
|
| 842 |
+
},
|
| 843 |
+
Ok(status) => {
|
| 844 |
+
let exit_code = status.code().unwrap_or(0);
|
| 845 |
+
|
| 846 |
+
if duration >= Duration::from_secs(timeout_secs) {
|
| 847 |
+
return TestResult {
|
| 848 |
+
name: test.name.clone(),
|
| 849 |
+
passed: false,
|
| 850 |
+
duration,
|
| 851 |
+
diff_lines: Vec::new(),
|
| 852 |
+
error: Some("Timeout".to_string()),
|
| 853 |
+
crashed: None,
|
| 854 |
+
};
|
| 855 |
+
}
|
| 856 |
+
|
| 857 |
+
// Crash detection (same as headless)
|
| 858 |
+
let errorlog_content = fs::read(&errorlog_path)
|
| 859 |
+
.ok()
|
| 860 |
+
.filter(|b| !b.is_empty())
|
| 861 |
+
.map(|b| String::from_utf8_lossy(&b).into_owned());
|
| 862 |
+
|
| 863 |
+
if is_crash_exit_code(exit_code) || errorlog_content.is_some() {
|
| 864 |
+
let unsigned = exit_code as u32;
|
| 865 |
+
let name = if is_crash_exit_code(exit_code) {
|
| 866 |
+
ntstatus_name(unsigned)
|
| 867 |
+
} else {
|
| 868 |
+
parse_errorlog_exception(errorlog_content.as_deref())
|
| 869 |
+
};
|
| 870 |
+
return TestResult {
|
| 871 |
+
name: test.name.clone(),
|
| 872 |
+
passed: false,
|
| 873 |
+
duration,
|
| 874 |
+
diff_lines: Vec::new(),
|
| 875 |
+
error: None,
|
| 876 |
+
crashed: Some(CrashInfo {
|
| 877 |
+
exit_code: unsigned,
|
| 878 |
+
name,
|
| 879 |
+
errorlog_content,
|
| 880 |
+
}),
|
| 881 |
+
};
|
| 882 |
+
}
|
| 883 |
+
|
| 884 |
+
// Analyze OpenWA.log for panics and gameplay markers
|
| 885 |
+
let analysis = analyze_headful_log(&openwa_log);
|
| 886 |
+
|
| 887 |
+
let mut detail_lines = Vec::new();
|
| 888 |
+
for line in &analysis.panics {
|
| 889 |
+
detail_lines.push(format!("PANIC: {line}"));
|
| 890 |
+
}
|
| 891 |
+
for line in &analysis.gameplay_fails {
|
| 892 |
+
detail_lines.push(format!("FAIL: {line}"));
|
| 893 |
+
}
|
| 894 |
+
for line in &analysis.gameplay_passes {
|
| 895 |
+
detail_lines.push(line.clone());
|
| 896 |
+
}
|
| 897 |
+
|
| 898 |
+
let passed = analysis.panics.is_empty()
|
| 899 |
+
&& analysis.gameplay_fails.is_empty()
|
| 900 |
+
&& !analysis.gameplay_passes.is_empty();
|
| 901 |
+
|
| 902 |
+
let error = if analysis.gameplay_passes.is_empty()
|
| 903 |
+
&& analysis.panics.is_empty()
|
| 904 |
+
&& analysis.gameplay_fails.is_empty()
|
| 905 |
+
{
|
| 906 |
+
Some(format!(
|
| 907 |
+
"No gameplay markers found in log (exit code: {exit_code})"
|
| 908 |
+
))
|
| 909 |
+
} else {
|
| 910 |
+
None
|
| 911 |
+
};
|
| 912 |
+
|
| 913 |
+
TestResult {
|
| 914 |
+
name: test.name.clone(),
|
| 915 |
+
passed,
|
| 916 |
+
duration,
|
| 917 |
+
diff_lines: detail_lines,
|
| 918 |
+
error,
|
| 919 |
+
crashed: None,
|
| 920 |
+
}
|
| 921 |
+
}
|
| 922 |
+
}
|
| 923 |
+
}
|
| 924 |
+
|
| 925 |
+
fn run_headful(args: HeadfulArgs) {
|
| 926 |
+
let tests = discover_headful_tests(args.filter_or_replay.as_deref());
|
| 927 |
+
if tests.is_empty() {
|
| 928 |
+
eprintln!("No replay tests found in {REPLAYS_DIR}/");
|
| 929 |
+
if let Some(f) = &args.filter_or_replay {
|
| 930 |
+
eprintln!(" (filter/path: \"{f}\")");
|
| 931 |
+
}
|
| 932 |
+
std::process::exit(1);
|
| 933 |
+
}
|
| 934 |
+
|
| 935 |
+
// Build
|
| 936 |
+
if !args.no_build {
|
| 937 |
+
eprint!("Building... ");
|
| 938 |
+
match build() {
|
| 939 |
+
Ok(d) => eprintln!("done ({:.1}s)", d.as_secs_f64()),
|
| 940 |
+
Err(e) => {
|
| 941 |
+
eprintln!("FAILED: {e}");
|
| 942 |
+
std::process::exit(1);
|
| 943 |
+
}
|
| 944 |
+
}
|
| 945 |
+
}
|
| 946 |
+
|
| 947 |
+
let wa_exe = find_wa_exe(args.wa_path.as_deref()).unwrap_or_else(|| {
|
| 948 |
+
eprintln!("Cannot find WA.exe. Use --wa-path or set OPENWA_WA_PATH.");
|
| 949 |
+
std::process::exit(1);
|
| 950 |
+
});
|
| 951 |
+
let launcher = find_launcher().unwrap_or_else(|| {
|
| 952 |
+
eprintln!("Cannot find openwa-launcher.exe. Run cargo build first.");
|
| 953 |
+
std::process::exit(1);
|
| 954 |
+
});
|
| 955 |
+
|
| 956 |
+
// Clean stale files from game dir
|
| 957 |
+
if let Some(game_dir) = wa_exe.parent() {
|
| 958 |
+
let _ = fs::remove_file(game_dir.join("ERRORLOG.TXT"));
|
| 959 |
+
let _ = fs::remove_file(game_dir.join("OpenWA.log"));
|
| 960 |
+
}
|
| 961 |
+
|
| 962 |
+
let timestamp = timestamp();
|
| 963 |
+
let run_dir = PathBuf::from(RUNS_DIR).join(format!("headful-{timestamp}"));
|
| 964 |
+
let _ = fs::create_dir_all(&run_dir);
|
| 965 |
+
let run_dir = strip_unc(fs::canonicalize(&run_dir).unwrap_or(run_dir));
|
| 966 |
+
|
| 967 |
+
println!(
|
| 968 |
+
"Running {} headful test{}...\n",
|
| 969 |
+
tests.len(),
|
| 970 |
+
if tests.len() == 1 { "" } else { "s" }
|
| 971 |
+
);
|
| 972 |
+
|
| 973 |
+
let wall_start = Instant::now();
|
| 974 |
+
let mut results = Vec::new();
|
| 975 |
+
|
| 976 |
+
for test in &tests {
|
| 977 |
+
let result = run_headful_test(test, &launcher, &wa_exe, &run_dir, args.timeout_secs);
|
| 978 |
+
print_result(&result);
|
| 979 |
+
results.push(result);
|
| 980 |
+
}
|
| 981 |
+
|
| 982 |
+
let wall_time = wall_start.elapsed();
|
| 983 |
+
|
| 984 |
+
print_summary(&results, wall_time);
|
| 985 |
+
write_summary(&results, wall_time, &run_dir.join("summary.txt"));
|
| 986 |
+
|
| 987 |
+
// Report startup check failures
|
| 988 |
+
report_startup_check_failures(&results, &run_dir);
|
| 989 |
+
|
| 990 |
+
cleanup_temp_files(&wa_exe);
|
| 991 |
+
|
| 992 |
+
let failed = results.iter().any(|r| !r.passed);
|
| 993 |
+
if failed {
|
| 994 |
+
std::process::exit(1);
|
| 995 |
+
}
|
| 996 |
+
}
|
| 997 |
+
|
| 998 |
// ─── Trace-desync subcommand ─────────────────────────────────────────────
|
| 999 |
|
| 1000 |
struct TraceDesyncArgs {
|
|
|
|
| 1323 |
fn main() {
|
| 1324 |
// Check for subcommands before normal arg parsing
|
| 1325 |
let argv: Vec<String> = env::args().skip(1).collect();
|
| 1326 |
+
match argv.first().map(|s| s.as_str()) {
|
| 1327 |
+
Some("headful") => {
|
| 1328 |
+
let sub_args = parse_headful_args(&argv[1..]);
|
| 1329 |
+
run_headful(sub_args);
|
| 1330 |
+
return;
|
| 1331 |
+
}
|
| 1332 |
+
Some("trace-desync") => {
|
| 1333 |
+
let sub_args = parse_trace_desync_args(&argv[1..]);
|
| 1334 |
+
run_trace_desync(sub_args);
|
| 1335 |
+
return;
|
| 1336 |
+
}
|
| 1337 |
+
_ => {}
|
| 1338 |
}
|
| 1339 |
|
| 1340 |
let args = parse_args();
|
replay-test.ps1
CHANGED
|
@@ -1,217 +1,6 @@
|
|
| 1 |
-
# replay
|
| 2 |
-
# Usage:
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
# Default (headful): Fast-forward replay with validation, graphics & sound active.
|
| 6 |
-
# -Headless: Uses WA's /getlog mode -- no window, pure simulation.
|
| 7 |
-
# Compares output log to expected log for determinism checking.
|
| 8 |
|
| 9 |
-
|
| 10 |
-
[switch]$Headless,
|
| 11 |
-
[string]$ReplayFile = "testdata\replays\bots.WAgame"
|
| 12 |
-
)
|
| 13 |
-
|
| 14 |
-
$ErrorActionPreference = "Stop"
|
| 15 |
-
|
| 16 |
-
# Resolve WA installation directory using the same priority order as openwa-config:
|
| 17 |
-
# 1. OPENWA_WA_PATH env var (may be WA.exe path or directory)
|
| 18 |
-
# 2. HKCU\SOFTWARE\Team17SoftwareLTD\WormsArmageddon, value PATH
|
| 19 |
-
function Get-WaDir {
|
| 20 |
-
if ($env:OPENWA_WA_PATH) {
|
| 21 |
-
$p = $env:OPENWA_WA_PATH
|
| 22 |
-
if ($p -match '\.exe$') { return Split-Path $p }
|
| 23 |
-
return $p
|
| 24 |
-
}
|
| 25 |
-
try {
|
| 26 |
-
$reg = Get-ItemPropertyValue -Path 'HKCU:\SOFTWARE\Team17SoftwareLTD\WormsArmageddon' -Name 'PATH' -ErrorAction Stop
|
| 27 |
-
if ($reg) { return $reg }
|
| 28 |
-
} catch {}
|
| 29 |
-
return $null
|
| 30 |
-
}
|
| 31 |
-
|
| 32 |
-
$gameDir = Get-WaDir
|
| 33 |
-
if (-not $gameDir) {
|
| 34 |
-
Write-Error "Cannot find WA installation. Set OPENWA_WA_PATH or ensure the registry key HKCU\SOFTWARE\Team17SoftwareLTD\WormsArmageddon\PATH is set."
|
| 35 |
-
exit 1
|
| 36 |
-
}
|
| 37 |
-
$waExe = Join-Path $gameDir "WA.exe"
|
| 38 |
-
$src = "target\i686-pc-windows-msvc\release"
|
| 39 |
-
$logDir = "testdata\logs"
|
| 40 |
-
|
| 41 |
-
# Resolve replay file to absolute path
|
| 42 |
-
if (-not [System.IO.Path]::IsPathRooted($ReplayFile)) {
|
| 43 |
-
$ReplayFile = Join-Path (Get-Location) $ReplayFile
|
| 44 |
-
}
|
| 45 |
-
|
| 46 |
-
if (-not (Test-Path $ReplayFile)) {
|
| 47 |
-
Write-Error "Replay file not found: $ReplayFile"
|
| 48 |
-
exit 1
|
| 49 |
-
}
|
| 50 |
-
|
| 51 |
-
# 1. Build launcher + DLL
|
| 52 |
-
Write-Host "Building..." -ForegroundColor Cyan
|
| 53 |
-
cargo build --release -p openwa-launcher -p openwa-wormkit
|
| 54 |
-
if ($LASTEXITCODE -ne 0) { exit 1 }
|
| 55 |
-
|
| 56 |
-
$launcher = "$src\openwa-launcher.exe"
|
| 57 |
-
|
| 58 |
-
# 2. Clear old logs
|
| 59 |
-
Remove-Item "$gameDir\OpenWA.log" -ErrorAction SilentlyContinue
|
| 60 |
-
Remove-Item "$gameDir\OpenWA_validation.log" -ErrorAction SilentlyContinue
|
| 61 |
-
Remove-Item "$gameDir\ERRORLOG.TXT" -ErrorAction SilentlyContinue
|
| 62 |
-
|
| 63 |
-
# Remove old WormKit DLL if present from a previous deployment
|
| 64 |
-
if (Test-Path "$gameDir\wkOpenWA.dll") {
|
| 65 |
-
Write-Host "Note: removing legacy $gameDir\wkOpenWA.dll" -ForegroundColor Yellow
|
| 66 |
-
Remove-Item "$gameDir\wkOpenWA.dll"
|
| 67 |
-
}
|
| 68 |
-
if (Test-Path "$gameDir\wkOpenWAValidator.dll") {
|
| 69 |
-
Remove-Item "$gameDir\wkOpenWAValidator.dll"
|
| 70 |
-
}
|
| 71 |
-
|
| 72 |
-
# 3. Launch via launcher with replay, env vars set
|
| 73 |
-
if ($Headless) {
|
| 74 |
-
Write-Host "Launching WA.exe in HEADLESS mode: $ReplayFile" -ForegroundColor Cyan
|
| 75 |
-
Write-Host " (No window -- pure simulation via /getlog)" -ForegroundColor Yellow
|
| 76 |
-
|
| 77 |
-
# Remove any previous output log next to the replay file
|
| 78 |
-
$replayBase = [System.IO.Path]::ChangeExtension($ReplayFile, ".log")
|
| 79 |
-
Remove-Item $replayBase -ErrorAction SilentlyContinue
|
| 80 |
-
|
| 81 |
-
$env:OPENWA_HEADLESS = "1"
|
| 82 |
-
$proc = Start-Process -FilePath $launcher -ArgumentList "`"$waExe`" /getlog `"$ReplayFile`"" -PassThru
|
| 83 |
-
$timeout = 120
|
| 84 |
-
} else {
|
| 85 |
-
Write-Host "Launching WA.exe with replay: $ReplayFile" -ForegroundColor Cyan
|
| 86 |
-
Write-Host " (Fast-forward mode: replay will be auto-advanced, game exits when done)" -ForegroundColor Yellow
|
| 87 |
-
|
| 88 |
-
$env:OPENWA_REPLAY_TEST = "1"
|
| 89 |
-
$proc = Start-Process -FilePath $launcher -ArgumentList "--minimized `"$waExe`" `"$ReplayFile`"" -PassThru
|
| 90 |
-
$timeout = 150
|
| 91 |
-
}
|
| 92 |
-
|
| 93 |
-
if (-not $proc.WaitForExit($timeout * 1000)) {
|
| 94 |
-
Write-Host "WARNING: launcher did not exit within ${timeout}s, killing..." -ForegroundColor Red
|
| 95 |
-
$proc.Kill()
|
| 96 |
-
$proc.WaitForExit(5000)
|
| 97 |
-
}
|
| 98 |
-
$env:OPENWA_REPLAY_TEST = $null
|
| 99 |
-
$env:OPENWA_HEADLESS = $null
|
| 100 |
-
$env:OPENWA_WA_PATH = $null
|
| 101 |
-
|
| 102 |
-
Write-Host "Launcher exited with code $($proc.ExitCode)" -ForegroundColor Cyan
|
| 103 |
-
|
| 104 |
-
# 4. Copy logs back
|
| 105 |
-
if (-not (Test-Path $logDir)) {
|
| 106 |
-
New-Item -ItemType Directory -Path $logDir | Out-Null
|
| 107 |
-
}
|
| 108 |
-
|
| 109 |
-
$timestamp = Get-Date -Format "yyyy-MM-dd_HH-mm-ss"
|
| 110 |
-
|
| 111 |
-
if (Test-Path "$gameDir\OpenWA_validation.log") {
|
| 112 |
-
Copy-Item "$gameDir\OpenWA_validation.log" "$logDir\validation_$timestamp.log"
|
| 113 |
-
Copy-Item "$gameDir\OpenWA_validation.log" "$logDir\validation_latest.log"
|
| 114 |
-
Write-Host "Validation log copied." -ForegroundColor Green
|
| 115 |
-
} else {
|
| 116 |
-
Write-Host "WARNING: No validation log found!" -ForegroundColor Red
|
| 117 |
-
}
|
| 118 |
-
|
| 119 |
-
if (Test-Path "$gameDir\OpenWA.log") {
|
| 120 |
-
Copy-Item "$gameDir\OpenWA.log" "$logDir\openwa_$timestamp.log"
|
| 121 |
-
Copy-Item "$gameDir\OpenWA.log" "$logDir\openwa_latest.log"
|
| 122 |
-
Write-Host "OpenWA log copied." -ForegroundColor Green
|
| 123 |
-
}
|
| 124 |
-
|
| 125 |
-
$crashed = $false
|
| 126 |
-
if (Test-Path "$gameDir\ERRORLOG.TXT") {
|
| 127 |
-
Copy-Item "$gameDir\ERRORLOG.TXT" "$logDir\errorlog_$timestamp.txt"
|
| 128 |
-
Copy-Item "$gameDir\ERRORLOG.TXT" "$logDir\errorlog_latest.txt"
|
| 129 |
-
Write-Host "ERRORLOG.TXT copied (crash detected)." -ForegroundColor Red
|
| 130 |
-
$crashed = $true
|
| 131 |
-
} else {
|
| 132 |
-
Remove-Item "$logDir\errorlog_latest.txt" -ErrorAction SilentlyContinue
|
| 133 |
-
}
|
| 134 |
-
|
| 135 |
-
# 5. Crash check (before summary, so it always runs)
|
| 136 |
-
if ($crashed) {
|
| 137 |
-
Write-Host ""
|
| 138 |
-
Write-Host "=== CRASH DETECTED ===" -ForegroundColor Red
|
| 139 |
-
Get-Content "$logDir\errorlog_latest.txt" -TotalCount 5 | ForEach-Object {
|
| 140 |
-
Write-Host " $_" -ForegroundColor Red
|
| 141 |
-
}
|
| 142 |
-
Write-Host " ..." -ForegroundColor Red
|
| 143 |
-
Write-Host " Full log: $logDir\errorlog_latest.txt" -ForegroundColor Gray
|
| 144 |
-
exit 1
|
| 145 |
-
}
|
| 146 |
-
|
| 147 |
-
# 6. Print summary
|
| 148 |
-
Write-Host ""
|
| 149 |
-
if ($Headless) {
|
| 150 |
-
Write-Host "=== Headless Log Comparison ===" -ForegroundColor Cyan
|
| 151 |
-
$replayBase = [System.IO.Path]::ChangeExtension($ReplayFile, ".log")
|
| 152 |
-
$expectedLog = [System.IO.Path]::Combine(
|
| 153 |
-
[System.IO.Path]::GetDirectoryName($ReplayFile),
|
| 154 |
-
[System.IO.Path]::GetFileNameWithoutExtension($ReplayFile) + "_expected.log"
|
| 155 |
-
)
|
| 156 |
-
|
| 157 |
-
if (-not (Test-Path $replayBase)) {
|
| 158 |
-
Write-Host " FAIL: No output log generated at $replayBase" -ForegroundColor Red
|
| 159 |
-
exit 1
|
| 160 |
-
}
|
| 161 |
-
if (-not (Test-Path $expectedLog)) {
|
| 162 |
-
Write-Host " No expected log found -- saving output as expected: $expectedLog" -ForegroundColor Yellow
|
| 163 |
-
Copy-Item $replayBase $expectedLog
|
| 164 |
-
Write-Host " PASS (first run -- expected log created)" -ForegroundColor Green
|
| 165 |
-
} else {
|
| 166 |
-
$diff = Compare-Object (Get-Content $expectedLog) (Get-Content $replayBase)
|
| 167 |
-
if ($diff) {
|
| 168 |
-
Write-Host " FAIL: Output differs from expected log" -ForegroundColor Red
|
| 169 |
-
Write-Host ""
|
| 170 |
-
$diff | ForEach-Object {
|
| 171 |
-
$indicator = if ($_.SideIndicator -eq "<=") { "expected" } else { "actual " }
|
| 172 |
-
Write-Host " $indicator | $($_.InputObject)" -ForegroundColor $(if ($_.SideIndicator -eq "<=") { "Red" } else { "Yellow" })
|
| 173 |
-
}
|
| 174 |
-
exit 1
|
| 175 |
-
} else {
|
| 176 |
-
Write-Host " PASS: Output matches expected log" -ForegroundColor Green
|
| 177 |
-
}
|
| 178 |
-
}
|
| 179 |
-
# Clean up the generated log
|
| 180 |
-
Remove-Item $replayBase -ErrorAction SilentlyContinue
|
| 181 |
-
} else {
|
| 182 |
-
Write-Host "=== Validation Summary ===" -ForegroundColor Cyan
|
| 183 |
-
if (Test-Path "$logDir\validation_latest.log") {
|
| 184 |
-
$content = Get-Content "$logDir\validation_latest.log" -Raw
|
| 185 |
-
|
| 186 |
-
# Static checks (vtables, struct offsets, prologues)
|
| 187 |
-
$staticPasses = ([regex]::Matches($content, "\[STATIC PASS\]")).Count
|
| 188 |
-
$staticFails = ([regex]::Matches($content, "\[STATIC FAIL\]")).Count
|
| 189 |
-
$staticColor = if ($staticFails -gt 0) { "Red" } else { "Green" }
|
| 190 |
-
Write-Host " Static checks: $staticPasses pass, $staticFails fail (vtables, struct offsets, prologues)" -ForegroundColor $staticColor
|
| 191 |
-
|
| 192 |
-
# Gameplay checks (milestones from frame hook)
|
| 193 |
-
$gameplayPasses = ([regex]::Matches($content, "\[GAMEPLAY PASS\]")).Count
|
| 194 |
-
$gameplayFails = ([regex]::Matches($content, "\[GAMEPLAY FAIL\]")).Count
|
| 195 |
-
if ($gameplayPasses + $gameplayFails -gt 0) {
|
| 196 |
-
$gameplayColor = if ($gameplayFails -gt 0) { "Red" } else { "Green" }
|
| 197 |
-
Write-Host " Gameplay checks: $gameplayPasses pass, $gameplayFails fail (init, match start, match end)" -ForegroundColor $gameplayColor
|
| 198 |
-
} else {
|
| 199 |
-
Write-Host " Gameplay checks: not found (DLL may not have detached cleanly)" -ForegroundColor Yellow
|
| 200 |
-
}
|
| 201 |
-
|
| 202 |
-
if ($content -match "Safety timeout reached") {
|
| 203 |
-
Write-Host " Replay: TIMEOUT (safety timeout triggered)" -ForegroundColor Red
|
| 204 |
-
}
|
| 205 |
-
}
|
| 206 |
-
# Check for Rust panics in OpenWA log
|
| 207 |
-
if (Test-Path "$logDir\openwa_latest.log") {
|
| 208 |
-
$panics = Select-String -Path "$logDir\openwa_latest.log" -Pattern "\[PANIC\]"
|
| 209 |
-
if ($panics) {
|
| 210 |
-
Write-Host ""
|
| 211 |
-
Write-Host "=== RUST PANIC DETECTED ===" -ForegroundColor Red
|
| 212 |
-
$panics | ForEach-Object { Write-Host " $($_.Line)" -ForegroundColor Red }
|
| 213 |
-
}
|
| 214 |
-
}
|
| 215 |
-
|
| 216 |
-
Write-Host " Logs: $logDir\" -ForegroundColor Gray
|
| 217 |
-
}
|
|
|
|
| 1 |
+
# Run a headful replay test via the Rust test runner.
|
| 2 |
+
# Usage: .\replay-test.ps1 [filter|replay.WAgame] [--no-build] [--wa-path PATH]
|
| 3 |
+
cargo build -p openwa-test-runner --release
|
| 4 |
+
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
|
|
|
|
|
|
|
|
|
|
| 5 |
|
| 6 |
+
& "$PSScriptRoot\target\i686-pc-windows-msvc\release\openwa-test.exe" headful @args
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|