README.md

---
license: cc-by-nc-4.0
language:
- en
pretty_name: ReactiveGWM-Models (SF2 / SF3)
library_name: diffusers
pipeline_tag: text-to-video
tags:
- video
- image-to-video
- world-model
- game-world-model
- fighting-game
- street-fighter
- behavior-cloning
- diffusion
---

# ReactiveGWM-Models: Reactive Game World Models for Street Fighter

<a href="https://inv-wzq.github.io/ReactiveGWM/"><img src="https://img.shields.io/badge/Web-Project Page-1d72b8.svg" alt="Project Page"></a>
<a href="http://arxiv.org/abs/2605.15256"><img src="https://img.shields.io/badge/arXiv-ReactiveGWM-A42C25.svg" alt="arXiv"></a>
<a href="https://github.com/INV-WZQ/ReactiveGWM"><img src="https://img.shields.io/badge/GitHub-Code-181717.svg?logo=github" alt="GitHub"></a>
<a href="https://huggingface.co/INV-WZQ/ReactiveGWM-Models"><img src="https://img.shields.io/badge/🤗_HuggingFace-Model-ffbd45.svg" alt="HuggingFace Model"></a>
<a href="https://huggingface.co/datasets/INV-WZQ/ReactiveGWM-Datasets"><img src="https://img.shields.io/badge/🤗_HuggingFace-Dataset-ffbd45.svg" alt="HuggingFace Dataset"></a>

## 📚 Models-Introduction

**ReactiveGWM-Models** is the model release accompanying **ReactiveGWM**, a
game world model that decouples *player control* from *NPC autonomy*.
Player actions are injected as a lightweight additive bias into the
diffusion backbone, while NPC behaviour is conditioned through dedicated
cross-attention modules over a structured `Active_Behavior /
Passive_Behavior / Strategy` prompt.

The weights here are trained on
[ReactiveGWM-Datasets](https://huggingface.co/datasets/INV-WZQ/ReactiveGWM-Datasets),
which provides 5-second strategy-aligned rollouts on two classic Capcom
fighting games:

| Subset | Game (platform)                                        |
|--------|--------------------------------------------------------|
| `SF2`  | Street Fighter II: Champion Edition (Sega Genesis)     |
| `SF3`  | Street Fighter III: New Generation / 3rd Strike (CPS3) |

For each game we release three checkpoints, all stored as `.safetensors`:

| File                              | Role                                                                                          |
|-----------------------------------|-----------------------------------------------------------------------------------------------|
| `ReactiveGWM_base.safetensors`    | Full ReactiveGWM trained from scratch on the structured strategy prompt + per-frame actions.  |
| `ReactiveGWM_transfer.safetensors`| ReactiveGWM initialised from a vanilla checkpoint of Game 1 and transfer-trained on the strategy prompt. |
| `Vanilla.safetensors`             | Vanilla world model baseline trained on the same clips with the free-form narration prompt (no NPC strategy decoupling). |

## 🗂️ Directory layout

```
SF2/
├── ReactiveGWM_base.safetensors
├── ReactiveGWM_transfer.safetensors
└── Vanilla.safetensors
SF3/   (same layout)
```

## 🎮 Conditioning interface

`ReactiveGWM_base` and `ReactiveGWM_transfer` expect **two parallel inputs**:

1. **Structured NPC prompt**, with the same shape used in the dataset's
   `metadata.csv`:

   ```
   NPC: Active_Behavior(<move>: <desc>; ...),
        Passive_Behavior(<move>: <desc>; ...),
        Strategy(Offense | Control | Defense : <desc>)
   ```

2. **Per-frame controller state** for both players (buttons + d-pad),
   aligned 1-to-1 with the output frames — the same schema as
   `actions.parquet` in the dataset. This stream is injected as a
   lightweight additive bias into the diffusion backbone.

`Vanilla.safetensors` instead consumes a **free-form English narration**
in the style of `metadata_vanilla.csv`, and serves as the text-to-video
baseline for ablations.

Inference / training code, configs, and the data-generation pipeline live
in a separate code release and are **not** distributed in this
repository.

## 🤓 Acknowledgments

Training data was recorded with the
[stable-retro](https://github.com/Farama-Foundation/stable-retro) framework
(SF3 via a CPS-3 custom integration on top of FBNeo). NPC strategy
annotations were produced by
[Gemini](https://deepmind.google/technologies/gemini/). The Street Fighter
games themselves are © Capcom Co., Ltd.; see the IP notice below. We
thank the open-source community for their valuable contributions.

## 🔗 Citation

If you use these models, please cite:

```bibtex
@misc{wang2026reactivegwmsteeringnpcreactive,
      title={ReactiveGWM: Steering NPC in Reactive Game World Models}, 
      author={Zeqing Wang and Danze Chen and Zhaohu Xing and Zizhao Tong and Yinhan Zhang and Xingyi Yang and Yeying Jin},
      year={2026},
      eprint={2605.15256},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2605.15256}, 
}
```

---

## ⚖️ Copyright, IP, and licensing

**Read this before using the models.**

These models were trained on derivative material from two commercial
video games whose copyright the model authors **do not own**:

- *Street Fighter II: Champion Edition* — © **Capcom Co., Ltd.**
- *Street Fighter III: New Generation* / *3rd Strike* — © **Capcom Co., Ltd.**

All in-game visuals, character designs, sprites, backgrounds, audio, and
game mechanics that the models can reproduce in their generated frames
remain the **exclusive property of Capcom Co., Ltd.** No ownership over
the underlying games is claimed or transferred by this release.

### What is and isn't licensed

The **CC BY-NC 4.0** license declared in the YAML header **applies only**
to the contributions that are original to this release:

- the trained model weights in `SF2/*.safetensors` and `SF3/*.safetensors`,
- the model architecture, training recipe choices, and accompanying
  documentation.

The **visual content these models can synthesise** — sprites, character
designs, stages, UI, audio, and any other recognisable Capcom IP —
remains © Capcom Co., Ltd. and is **not** relicensed by this release.
The weights are made available **solely under fair-use / fair-dealing
for non-commercial academic research** on game AI, world models, and
video understanding. CC BY-NC 4.0 does **not** grant you any right to
the underlying Capcom IP. See the `LICENSE` and `NOTICE` files shipped
alongside this README for the full statement.

### What you may NOT do

- Use the weights, generated outputs, or any model derived from them in
  a **commercial product**, including a game, a streaming service, or a
  paid API.
- Use the models, or any further fine-tune of them, to **reproduce,
  recreate, clone, or compete with** Capcom's *Street Fighter* games or
  any other Capcom title.
- Redistribute the weights bundled with **game ROMs**, BIOS files, or
  emulator binaries.
- Claim ownership of, or relicense, any Capcom IP that the models
  reproduce in their outputs.

### What you should do

- Treat this as an **academic, non-commercial research artifact only**.
- Cite both this model release *and* acknowledge Capcom as the copyright
  holder of the source games in any publication or release that uses it.
- If you publish a downstream model or a generated-content release built
  on these weights, **carry forward the same non-commercial restriction
  and copyright notice**.

### ROMs

**No game ROMs are included in this repository, and none will be
provided.** Reproducing the training data from the upstream generation
code requires you to obtain legal copies of the original Street Fighter
II (Genesis) and Street Fighter III (CPS3) ROMs yourself, in accordance
with the laws of your jurisdiction.

### Takedown / DMCA

If you are a rights holder (in particular Capcom Co., Ltd. or an
authorised representative) and believe these model weights or any
content they reproduce infringes your rights, please contact
**zeqing.wang@u.nus.edu** with a description of the material in
question. The maintainer will respond and, where appropriate, remove
the contested content promptly.