Instructions to use wuzhi-hao/EgoSim with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use wuzhi-hao/EgoSim with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("wuzhi-hao/EgoSim", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Add model card, pipeline tag, and links to paper/code
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,7 +1,76 @@
|
|
| 1 |
---
|
| 2 |
-
license: mit
|
| 3 |
base_model:
|
| 4 |
- alibaba-pai/Wan2.1-Fun-14B-InP
|
|
|
|
|
|
|
| 5 |
tags:
|
| 6 |
- World
|
| 7 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
base_model:
|
| 3 |
- alibaba-pai/Wan2.1-Fun-14B-InP
|
| 4 |
+
license: mit
|
| 5 |
+
pipeline_tag: image-to-video
|
| 6 |
tags:
|
| 7 |
- World
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
# EgoSim: Egocentric World Simulator for Embodied Interaction Generation
|
| 11 |
+
|
| 12 |
+
This repository contains the weights for **EgoSim-14B**, a closed-loop egocentric world simulator presented in the paper [EgoSim: Egocentric World Simulator for Embodied Interaction Generation](https://huggingface.co/papers/2604.01001).
|
| 13 |
+
|
| 14 |
+
[**Project Page**](https://egosimulator.github.io/) | [**GitHub Code**](https://github.com/jinkun-hao/EgoSim) | [**Paper**](https://huggingface.co/papers/2604.01001)
|
| 15 |
+
|
| 16 |
+

|
| 17 |
+
|
| 18 |
+
## Overview
|
| 19 |
+
**EgoSim** is an egocentric world simulator for embodiment interaction generation. Given an initial 3D state and a sequence of actions, EgoSim generates temporally and spatially consistent egocentric observations with high-quality dexterous interactions. EgoSim also persistently updates a 3D scene state for continuous simulation.
|
| 20 |
+
|
| 21 |
+
## Quickstart
|
| 22 |
+
|
| 23 |
+
### Installation
|
| 24 |
+
|
| 25 |
+
Requires Python 3.10+ and CUDA 12.1+.
|
| 26 |
+
|
| 27 |
+
```bash
|
| 28 |
+
git clone https://github.com/jinkun-hao/EgoSim.git
|
| 29 |
+
cd EgoSim
|
| 30 |
+
|
| 31 |
+
conda create -n egosim python=3.10 -y
|
| 32 |
+
conda activate egosim
|
| 33 |
+
|
| 34 |
+
# Install PyTorch
|
| 35 |
+
pip install torch torchvision
|
| 36 |
+
# Install flash attention
|
| 37 |
+
pip install flash-attn --no-build-isolation
|
| 38 |
+
pip install -r requirements.txt
|
| 39 |
+
```
|
| 40 |
+
|
| 41 |
+
### Model Download
|
| 42 |
+
|
| 43 |
+
Download the **EgoSim-14B** weights into the project root:
|
| 44 |
+
|
| 45 |
+
```bash
|
| 46 |
+
huggingface-cli download wuzhi-hao/EgoSim --local-dir ./EgoSim-14B
|
| 47 |
+
```
|
| 48 |
+
|
| 49 |
+
### Inference
|
| 50 |
+
|
| 51 |
+
Download the demo samples from the [official repository instructions](https://github.com/jinkun-hao/EgoSim), extract under `tests/samples/`, and run:
|
| 52 |
+
|
| 53 |
+
```bash
|
| 54 |
+
# Egodex — quick test with bundled mini samples
|
| 55 |
+
PYTHONPATH=. python egowm/inference/runner.py \
|
| 56 |
+
--dataset egodex \
|
| 57 |
+
--model_root ./EgoSim-14B \
|
| 58 |
+
--dataset_root tests/samples/demo_data/egodex \
|
| 59 |
+
--metadata_path tests/samples/demo_data/egodex_metadata.csv \
|
| 60 |
+
--output_dir output_egodex \
|
| 61 |
+
--num_inference_steps 50 \
|
| 62 |
+
--gpu_id 0
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
## Citation
|
| 66 |
+
|
| 67 |
+
If you find EgoSim useful in your research, please cite the paper:
|
| 68 |
+
|
| 69 |
+
```bibtex
|
| 70 |
+
@article{hao2026egosim,
|
| 71 |
+
title={EgoSim: Egocentric World Simulator for Embodied Interaction Generation},
|
| 72 |
+
author={Hao, Jinkun and Jia, Mingda and Wang, Ruiyan and Liu, Xihui and Yi, Ran and Ma, Lizhuang and Pang, Jiangmiao and Xu, Xudong},
|
| 73 |
+
journal={arXiv preprint arXiv:2604.01001},
|
| 74 |
+
year={2026}
|
| 75 |
+
}
|
| 76 |
+
```
|