Data

PACTBench expects the three game subsets distributed under the dataset identifier teawhite/vrising. Check the dataset card and its current terms before redistributing any raw game footage; this repository documents the layout used by the experiments but does not itself grant rights to third-party game content.

Processed layout

data/processed/<Boss>/                 # V Rising
data/hk/processed/<Boss>/              # Hollow Knight
data/isaac/processed/<Boss>/           # Isaac

<Boss>/
├── video_fight<N>.mp4
├── policy_view_fight<N>.jsonl
├── render_view_fight<N>.jsonl
└── dataset_manifest.json

Each policy_view row is a boss decision with an action and observation. Each render_view row supplies the aligned rendered prompt. Video, policy, and render files are audited together by boss and fight.

Fight-disjoint splits

Game	Bosses	Train fights	Validation fights	Test fights	Official test rows
V Rising	3	54	9	9	8,075
Hollow Knight	3	484	9	30	8,360
Isaac	8	4,858	24	80	1,784

Splits operate on complete fights. No fight appears in more than one split. V Rising uses the last three fights per boss for test and the preceding three for validation. Hollow Knight and Isaac use the last ten fights per boss for test and the preceding three for validation.

V Rising has two manifests:

games/vrising/manifest.json records the fight-level dataset split.
games/vrising/manifest_latent_covered.json is the canonical L1 training manifest and retains only fights with the required visual coverage.

Observation fields and units

Common policy observations include distance, angle, relative geometry, previous-action timing, fight time, and boss/player state. The exact schema is checked by the dataset audit and the belief validators.

V Rising and Hollow Knight distance use engine units.
Isaac obs.dp uses pixels (observed range 24--484; median about 147 px).
Isaac L1 therefore fixes distance_norm=1000 and distance_clamp=1.5.

Build and audit

Run from the pactbench root:

bash games/vrising/data/audit_dataset.sh
bash games/vrising/data/build_manifest.sh

bash games/hk/data/audit_dataset.sh
python3 games/hk/data/build_metadata.py
bash games/hk/data/build_manifest.sh

bash games/isaac/data/audit_dataset.sh
python3 games/isaac/data/build_metadata.py
bash games/isaac/data/build_manifest.sh

The audit outputs are:

results/vrising/dataset_audit.json
results/hk/dataset_audit.json
results/isaac/dataset_audit.json

HK and Isaac metadata are derived from the processed policy/render rows. V Rising uses the checked-in canonical ontology.json and skill_library.json.