Add model card with image-to-video pipeline tag and quick start guide

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +73 -0
README.md ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: image-to-video
3
+ ---
4
+
5
+ # HunyuanVideo-HOMA
6
+
7
+ HOMA (Towards Generic Human-Object Interaction in Multimodal Driven Human Animation with Weak Conditions) is a weakly conditioned multimodal-driven framework that turns a reference person image, an object image, speech audio, sparse motion, and a text prompt into a physically plausible human-object interaction video.
8
+
9
+ * **Paper:** [HunyuanVideo-HOMA: Generic Human-Object Interaction in Multimodal Driven Human Animation](https://huggingface.co/papers/2506.08797)
10
+ * **Project Page:** [https://bone-11.github.io/homa-page/](https://bone-11.github.io/homa-page/)
11
+ * **Repository:** [https://github.com/bone-11/homa](https://github.com/bone-11/homa)
12
+
13
+ ---
14
+
15
+ ## Quick Start (Inference)
16
+
17
+ To use these checkpoints, please clone the official GitHub repository and install the dependencies.
18
+
19
+ ### 1. Environment Setup
20
+
21
+ ```bash
22
+ conda create -n homa python=3.10 -y
23
+ conda activate homa
24
+
25
+ conda install -c conda-forge ffmpeg -y
26
+
27
+ # Match the CUDA wheel to your machine (e.g., CUDA 12.4)
28
+ pip install "torch>=2.4.0" torchvision --index-url https://download.pytorch.org/whl/cu124
29
+
30
+ pip install flash-attn --no-build-isolation
31
+ pip install -r requirements-infer.txt
32
+ ```
33
+
34
+ ### 2. Download Model Weights
35
+
36
+ First, download the public dependencies:
37
+ ```bash
38
+ bash download_models.sh
39
+ ```
40
+
41
+ Then, download the HOMA checkpoint from this repository:
42
+ ```bash
43
+ huggingface-cli download ProAudience/homa_checkpoint --local-dir checkpoints/homa_checkpoint
44
+ ```
45
+
46
+ ### 3. Run Inference
47
+
48
+ Configure the paths in `run_inference.sh`:
49
+ ```bash
50
+ CHECKPOINT_DIR="checkpoints/homa_checkpoint"
51
+ MODEL_BASE="checkpoints/other_models/pretrained_models"
52
+ MODEL_AUX_PATH="checkpoints/other_models/aux"
53
+ ```
54
+
55
+ Then run the demo:
56
+ ```bash
57
+ bash run_inference.sh
58
+ ```
59
+
60
+ For more detailed setup and running customized inference cases, please refer to the [GitHub repository](https://github.com/bone-11/homa).
61
+
62
+ ## Citation
63
+
64
+ If you find HOMA useful for your research, please cite the paper:
65
+
66
+ ```bibtex
67
+ @inproceedings{huang2025homa,
68
+ title = {HOMA: Towards Generic Human-Object Interaction in Multimodal Driven Human Animation with Weak Conditions},
69
+ author = {Huang, Ziyao and Zhou, Zixiang and Cao, Juan and Ma, Yifeng and Chen, Yi and Rao, Zejing and Xu, Zhiyong and Wang, Hongmei and Lin, Qin and Zhou, Yuan folding and Lu, Qinglin and Tang, Fan},
70
+ booktitle = {SIGGRAPH Asia 2025 Conference Papers},
71
+ year = {2025},
72
+ }
73
+ ```