metadata
pipeline_tag: image-to-video
HOMA (Siggraph Asia 2025)
HOMA (Towards Generic Human-Object Interaction in Multimodal Driven Human Animation with Weak Conditions) is a weakly conditioned multimodal-driven framework that turns a reference person image, an object image, speech audio, sparse motion, and a text prompt into a physically plausible human-object interaction video.
- Paper: HOMA: Towards Generic Human-Object Interaction in Multimodal Driven Human Animation with Weak Conditions
- Project Page: https://bone-11.github.io/homa-page/
- Repository: https://github.com/bone-11/homa
Quick Start (Inference)
To use these checkpoints, please clone the official GitHub repository and install the dependencies.
1. Environment Setup
conda create -n homa python=3.10 -y
conda activate homa
conda install -c conda-forge ffmpeg -y
# Match the CUDA wheel to your machine (e.g., CUDA 12.4)
pip install "torch>=2.4.0" torchvision --index-url https://download.pytorch.org/whl/cu124
pip install flash-attn --no-build-isolation
pip install -r requirements-infer.txt
2. Download Model Weights
First, download the public dependencies:
bash download_models.sh
Then, download the HOMA checkpoint from this repository:
huggingface-cli download ProAudience/homa_checkpoint --local-dir checkpoints/homa_checkpoint
3. Run Inference
Configure the paths in run_inference.sh:
CHECKPOINT_DIR="checkpoints/homa_checkpoint"
MODEL_BASE="checkpoints/other_models/pretrained_models"
MODEL_AUX_PATH="checkpoints/other_models/aux"
Then run the demo:
bash run_inference.sh
For more detailed setup and running customized inference cases, please refer to the GitHub repository.
Citation
If you find HOMA useful for your research, please cite the paper:
@inproceedings{huang2025homa,
title = {HOMA: Towards Generic Human-Object Interaction in Multimodal Driven Human Animation with Weak Conditions},
author = {Huang, Ziyao and Zhou, Zixiang and Cao, Juan and Ma, Yifeng and Chen, Yi and Rao, Zejing and Xu, Zhiyong and Wang, Hongmei and Lin, Qin and Zhou, Yuan folding and Lu, Qinglin and Tang, Fan},
booktitle = {SIGGRAPH Asia 2025 Conference Papers},
year = {2025},
}