nielsr HF Staff commited on
Commit
4e4ac4c
·
verified ·
1 Parent(s): df5f843

Add model card for HyDRA

Browse files

Hi! I'm Niels from the Hugging Face community science team. I noticed this repository doesn't have a model card yet. I've opened this PR to add a README that includes metadata, links to the paper and code, and usage instructions based on your GitHub repository.

The `text-to-video` pipeline tag has been added to the metadata to improve discoverability.

Files changed (1) hide show
  1. README.md +58 -0
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: text-to-video
4
+ ---
5
+
6
+ # HyDRA: Out of Sight but Not Out of Mind
7
+
8
+ This repository contains the weights for **HyDRA** (Hybrid Memory for Dynamic Video World Models), as presented in the paper [Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models](https://arxiv.org/abs/2603.25716).
9
+
10
+ HyDRA is a novel memory architecture that enables video world models to simultaneously act as precise archivists for static backgrounds and vigilant trackers for dynamic subjects. This ensures visual and motion continuity even when subjects temporarily move out of the camera's field of view.
11
+
12
+ [**Project Page**](https://kj-chen666.github.io/Hybrid-Memory-in-Video-World-Models/) | [**GitHub**](https://github.com/H-EmbodVis/HyDRA)
13
+
14
+ ## Installation
15
+
16
+ To set up the environment, clone the repository and install the dependencies:
17
+
18
+ ```bash
19
+ git clone https://github.com/H-EmbodVis/HyDRA.git
20
+ cd HyDRA
21
+ conda create -n hydra python=3.10 -y
22
+ conda activate hydra
23
+ pip install -r requirements.txt
24
+ ```
25
+
26
+ ## Inference
27
+
28
+ HyDRA is built upon the **Wan2.1 (1.3B) T2V** model. Ensure you have downloaded the required weights into the `./ckpts` directory as described in the official repository.
29
+
30
+ You can run inference on example data using the following command:
31
+
32
+ ```bash
33
+ python infer_hydra.py
34
+ ```
35
+
36
+ ## Training
37
+
38
+ The model can be trained on custom datasets using the provided training script:
39
+
40
+ ```bash
41
+ python train_hydra.py \
42
+ --dit_path ./ckpts/Wan2.1-T2V-1.3B/diffusion_pytorch_model.safetensors \
43
+ --use_gradient_checkpointing \
44
+ --hydra
45
+ ```
46
+
47
+ ## Citation
48
+
49
+ If you find this work useful, please consider citing:
50
+
51
+ ```bibtex
52
+ @article{chen2026out,
53
+ title = {Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models},
54
+ author = {Chen, Kaijin and Liang, Dingkang and Zhou, Xin and Ding, Yikang and Liu, Xiaoqiang and Wan, Pengfei and Bai, Xiang},
55
+ journal = {arXiv preprint arXiv:2603.25716},
56
+ year = {2026}
57
+ }
58
+ ```