README.md

---
pipeline_tag: image-to-video
---

# HOMA  (Siggraph Asia 2025)

HOMA (Towards Generic Human-Object Interaction in Multimodal Driven Human Animation with Weak Conditions) is a weakly conditioned multimodal-driven framework that turns a reference person image, an object image, speech audio, sparse motion, and a text prompt into a physically plausible human-object interaction video.

* **Paper:** [HOMA: Towards Generic Human-Object Interaction in Multimodal Driven Human Animation with Weak Conditions](https://huggingface.co/papers/2506.08797)
* **Project Page:** [https://bone-11.github.io/homa-page/](https://bone-11.github.io/homa-page/)
* **Repository:** [https://github.com/bone-11/homa](https://github.com/bone-11/homa)

---

## Quick Start (Inference)

To use these checkpoints, please clone the official GitHub repository and install the dependencies.

### 1. Environment Setup

```bash
conda create -n homa python=3.10 -y
conda activate homa

conda install -c conda-forge ffmpeg -y

# Match the CUDA wheel to your machine (e.g., CUDA 12.4)
pip install "torch>=2.4.0" torchvision --index-url https://download.pytorch.org/whl/cu124

pip install flash-attn --no-build-isolation
pip install -r requirements-infer.txt
```

### 2. Download Model Weights

First, download the public dependencies:
```bash
bash download_models.sh
```

Then, download the HOMA checkpoint from this repository:
```bash
huggingface-cli download ProAudience/homa_checkpoint --local-dir checkpoints/homa_checkpoint
```

### 3. Run Inference

Configure the paths in `run_inference.sh`:
```bash
CHECKPOINT_DIR="checkpoints/homa_checkpoint"
MODEL_BASE="checkpoints/other_models/pretrained_models"
MODEL_AUX_PATH="checkpoints/other_models/aux"
```

Then run the demo:
```bash
bash run_inference.sh
```

For more detailed setup and running customized inference cases, please refer to the [GitHub repository](https://github.com/bone-11/homa).

## Citation

If you find HOMA useful for your research, please cite the paper:

```bibtex
@inproceedings{huang2025homa,
  title     = {HOMA: Towards Generic Human-Object Interaction in Multimodal Driven Human Animation with Weak Conditions},
  author    = {Huang, Ziyao and Zhou, Zixiang and Cao, Juan and Ma, Yifeng and Chen, Yi and Rao, Zejing and Xu, Zhiyong and Wang, Hongmei and Lin, Qin and Zhou, Yuan folding and Lu, Qinglin and Tang, Fan},
  booktitle = {SIGGRAPH Asia 2025 Conference Papers},
  year      = {2025},
}
```