---
license: cc-by-4.0
tags:
  - world-models
  - robotics
  - research-notes
  - survey
---

# World Model Research Notes

> A living research knowledge base on **world models** — from gameplay/video data to robot control.
> Maintained by **[HakkoLab](https://huggingface.co/HakkoLab)** / **[Oratis](https://huggingface.co/Oratis)**.

This repo collects our running survey of the world-model literature and the methods most relevant to
learning interactive world models from gameplay/video and transferring them to robot autonomy.

## Contents

| File | What it is |
|------|-----------|
| [`world_models_survey.md`](world_models_survey.md) | SOTA survey (2024–2026): the four paradigms (autoregressive / diffusion / JEPA-latent / world-action), per-model deep dives (WHAM, V-JEPA 2, Genie 3, Cosmos, GameNGen/DIAMOND/Oasis, Dreamer…), open problems. |
| [`papers.md`](papers.md) | Annotated reading list with arXiv links, grouped and flagged (must-read / robotics / game-data). |
| [`latent_action_cross_embodiment.md`](latent_action_cross_embodiment.md) | Deep dive on **latent action** & cross-embodiment transfer (LAPA, Genie LAM, UniSkill, Latent Action Diffusion, V-JEPA 2-AC) — the bridge from unlabeled gameplay/video to robot actions. |
| [`training_plan.md`](training_plan.md) | Our staged training approach — VQ tokenizer → autoregressive world model (WHAM-style) → latent-action + controllable latent dynamics → robot transfer. Architecture choices, eval, compute anchors. |
| [`robotics_transfer.md`](robotics_transfer.md) | Three routes from a world model to robot autonomy (action-conditioned planning + MPC / representation backbone / dreamed policy training), the embodiment gap, and a recommended path. |
| [`world_model_benchmarks.md`](world_model_benchmarks.md) | Catalog of world-model benchmarks (2024–2026) across 6 categories — unified world-generation (WorldScore), physical reasoning (Physics-IQ, VideoPhy), action controllability (ACT-Bench), embodied/robotics (EWMBench, RoboWM-Bench), model-based RL (Atari 100k, DMC), and cross-cutting metrics. |

## Notes

- This is a **curated public subset** of a larger internal research effort. Product-specific and
  operational details are intentionally not included; the focus here is the general method and survey.
- Sources are cited inline (arXiv IDs are the stable anchors). Last refreshed **2026-06**.
- Updated periodically — a weekly tracker appends newly published, relevant papers to `papers.md`.

## License

Text released under **CC BY 4.0**. Cited papers belong to their respective authors.