license: apache-2.0
language:
- en
tags:
- video-generation
- human-object-interaction
- sparse-control
SparseCtrl-HOI-14B
Official checkpoints for SparseCtrl-HOI: Sparse Temporal Control for Human-Object Interaction Video Generation.
SparseCtrl-HOI generates human-object interaction videos conditioned on a reference image, audio, and sparse interaction keyframes with designated timestamps.
Paper · Project Page · Code
Checkpoints
stage1.ckpt: Stage-1 weights (LoRA and TiRoPE input adapter).stage2.ckpt: Stage-2 weights (Motion Prior Injection Module).prompt_emb.pt: Precomputed T5 embedding of the positive prompt below.prompt_emb_negative.pt: Precomputed T5 embedding of the negative prompt below.
Positive prompt
A person is introducing a product.
Negative prompt
Vivid color tones, background/camera moving quickly, screen switching, subtitles and special effects, mutation, overexposed, static, blurred details, subtitles, style, work, painting, image, still, overall grayish, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn face, deformed, disfigured, malformed limbs, fingers merging, motionless image, chaotic background, three legs, crowded background with many people, walking backward
Both stage checkpoints are required for inference. These files are not a standalone model; the additional pretrained models listed in the code repository are also required.
Usage
From the SparseCtrl-HOI project root, download the checkpoints:
hf download Moon-jack/SparseCtrl-HOI-14B \
--local-dir checkpoints/SparseCtrl-HOI-14B
Follow the GitHub README to install dependencies, download the additional models and run.
License
These checkpoints are released under the Apache License 2.0. Third-party models remain subject to their respective licenses.