File size: 6,589 Bytes

---
license: cc-by-nc-4.0
tags:
  - world-model
  - video-generation
  - diffusion
  - flow-matching
  - counter-strike
  - reinforcement-learning-environment
library_name: pytorch
---

# CS2 Latent World Model (MIRA-style)

An **action-conditioned world model for Counter-Strike 2** — feed it keyboard + mouse actions and it
imagines the next frames. Built by adapting [MIRA](https://github.com/mira-wm/mira) to stream
[RekaAI/CS2-10k](https://huggingface.co/datasets/RekaAI/CS2-10k) (63 TB) with **no local dataset copy**.

<p align="center">
  <img src="assets/walkthrough.gif" alt="Walking around inside the world model" width="640"/>
</p>

<a href="https://hfviewer.com/Quazim0t0/CS2Dream?utm_source=huggingface&amp;utm_medium=embedded_model_card&amp;utm_campaign=Quazim0t0__CS2Dream_card&amp;utm_content=embedded_card_open_viewer&amp;from=embedded-model-card" target="_blank" rel="noopener">
  <img
    src="https://hfviewer.com/api/card.svg?source=Quazim0t0%2FCS2Dream&amp;granularity=auto&amp;v=20260516-title-pills-card"
    alt="Architecture graph for Quazim0t0/CS2Dream. Open in hfviewer"
    width="100%"
  />
</a>

*Driving the world model with "hold W + sweep the mouse" — every frame above is **generated**, not real gameplay.*

> **Status: research prototype.** It works end-to-end (coherent, action-responsive rollouts) but is
> small, blurry, and trained on very little data. Limitations are documented honestly below.

## Real vs. generated

| Real gameplay (top) vs. world model (bottom) |
|---|
| ![real vs generated](assets/real_vs_generated_final.png) |

| Codec reconstruction — ground truth ‖ reconstruction |
|---|
| ![codec reconstruction](assets/codec_reconstruction.png) |

## Architecture

**Stage 1 — Video codec (RAEv2)**
- Frozen **DINOv3-L/16** (`facebook/dinov3-vitl16-pretrain-lvd1689m`) backbone, layer aggregation
  over blocks `[11,13,15,17,19,21,23]`.
- Strided-conv bottleneck → **32-channel latent at /32 spatial, /2 temporal**.
- ViT decoder (width 512, depth 6, patch 16). ~331 M params, mostly frozen.

**Stage 2 — Latent world model**
- Causal **flow-matching diffusion transformer** over codec latents — **12.5 M trainable** params
  (hidden 384, 4 layers, 6 heads).
- AdaLN conditioning on **per-frame keyboard + mouse** actions; diffusion forcing; action dropout 0.1.
- Autoregressive rollout with a streaming KV-cache.

**IO:** 288×512 @ 24 fps, 16-frame clips (CS2 is 48 fps → integer /2 downsample).

## Action space

10 keys — `W A S D J C R V [ ]` (move, jump, crouch, walk, freefall, fire, scope) as a multi-hot
vector, **plus continuous mouse deltas**.

> Upstream MIRA conditions on *key presses only* (Rocket League is keyboard-only). CS2 is
> mouse-driven, so mouse deltas are populated here and consumed by MIRA's existing `ActionEncoder`
> (symlog-normalized + temporally pooled).

## Files

| File | What |
|---|---|
| `codec_069000.pt` | Codec weights (decoder + bottleneck; frozen DINOv3 rebuilt from HF) |
| `world_model_030000.pt` | World model weights (12.5 M) |
| `scripts/`, `src/`, `configs/` | Training / eval / streaming code |
| `patches/dino_hf_backbone.patch` | Load gated DINOv3 via `transformers` instead of Meta's `.pth` |


## Training

| | Codec | World model |
|---|---|---|
| Steps | 69,000 | 30,000 |
| Hardware | RTX 5060 Ti (16 GB) | RTX 5060 Ti |
| Loss | L1 reconstruction | flow-matching diffusion |
| Final | ema 0.1392 | ema 2.41 |

Trained on the untarred `sample/` split: **3 matches / 374 clips / 44 (match, round) groups.**

## Results

- **Codec reconstruction: 24.75 dB PSNR** (vs 19.7 dB with a random backbone). Recovers room
  geometry and recognizable player figures; no fine texture.
- **Rollouts stay coherent** — players persist across the clip, room geometry is stable.
- **Controllability** — mean abs pixel difference of a generated rollout vs an idle-action rollout:
  - `forward` vs `idle`: **0.025 – 0.037**
  - `turn` vs `idle`: **0.040**
  - Non-zero **and growing over the clip** → the model integrates actions over time rather than
    ignoring them. This was the go/no-go test, and it passed.

## Limitations

- **Blurry.** The /32, 32-channel latent + a depth-6 decoder caps detail (no faces, weapon detail,
  floor texture). The paper uses a ViT-XL decoder.
- **Long-horizon drift.** Coherent for several frames past context, then degrades — dissolving to
  generic wall texture or darkening. 16-frame clips vs the paper's 80.
- **Tiny dataset — 3 matches.** The dominant constraint; there's little to generalize from.
- **Tiny model** — 12.5 M vs the paper's 1 B. Long-horizon coherence is what capacity buys.
- **Single-perspective only.** MIRA's defining contribution is *multiplayer* (joint, time-aligned
  multi-POV with per-player action attribution). Only the single-player stage is trained here. The
  multi-POV data path exists (`cs2_stream` groups POVs by `(match_id, round_number)`, with ranged
  tar-member fetch for the full split) but the joint stage was never trained.
- **Action semantics unverified.** We measure that actions *change* the output, not that "forward"
  moves precisely forward — fidelity is too low to confirm direction visually.

## Intended use / out of scope

Research on world models and action-conditioned video prediction. **Not** a game, an anti-cheat
signal, a player model, or anything deployable. Do not present generated frames as real gameplay.

## Attribution & licensing

- **MIRA** — architecture/training code (upstream; this repo ships patches + new modules).
- **DINOv3** (Meta) — frozen backbone, **gated weights** under Meta's license (not redistributed here).
- **CS2-10k** (RekaAI) — **CC BY-NC 4.0**.
- Counter-Strike 2 imagery © Valve; generated frames derive from that footage.

**Non-commercial research use only.**

## Usage

```bash
export RS_DINO_HF=facebook/dinov3-vitl16-pretrain-lvd1689m   # requires HF access to gated DINOv3
python scripts/consolidate_codec.py     # pack codec_069000.pt for the world model
python scripts/train_cs2_wm.py --test   # controllability check (generates rollouts)
python scripts/eval_cs2_codec.py        # reconstruction PSNR + side-by-side frames
```

<!-- CITE_START -->
## Citation

If you use this model, please cite:

```bibtex
@misc{escarda86mbase,
  title        = {CS2Dream: Counterstrike small latent world model},
  author       = {Dean Byrne (Quazim0t0)},
  year         = {2026},
  howpublished = {HuggingFace, \url{https://huggingface.co/Quazim0t0/CS2Dream}},
  note         = {Quazim0t0/CS2Dream}
}
```
<!-- CITE_END -->