---
language:
- en
license: other
license_name: nvidia-open-model-agreement
license_link: LICENSE
base_model:
- nvidia/MotionBricks
library_name: diffusers
tags:
- motionbricks
- modular-diffusers
- motion-generation
- motion-inbetweening
- soma
---

# MotionBricks SOMA

<video controls loop muted playsinline src="https://huggingface.co/datasets/OzzyGT/diffusers-examples/resolve/main/motionbricks_soma_diffusers/motionbricks_soma_run_stop.mp4"></video>

*The start of a Kimodo run, then the 30 frames generated with the code below, then the standing end of another Kimodo take, placed 1.4 m ahead. The generated frames are drawn on SOMA77 with a relaxed finger pose. At normal playback speed; the cameras follow the root.*

This is the SOMA model of [nvidia/MotionBricks](https://huggingface.co/nvidia/MotionBricks), repacked into the Diffusers format in float32. It runs as a Modular Diffusers pipeline with the [MotionBricks SOMA custom blocks](https://huggingface.co/OzzyGT/motionbricks_soma_custom_blocks) and generates the transition from the last poses of one motion to target poses in one forward pass, choosing the length of the transition itself.

The three components work together:

- `root_model` (40M parameters) chooses the length and predicts the root trajectory.
- `pose_model` (137M parameters) predicts the pose codes of every token of four frames.
- `pose_decoder` (12M parameters) decodes the codes to frames.

Note: You need `diffusers>=0.41.0`.

## Usage

The custom blocks load the model code shipped in this repository, hence `trust_remote_code` and `trust_components_code`.

**Note:** both flags run Python code downloaded from the Hub: `trust_remote_code=True` runs the code of the blocks repository, and `trust_components_code=True` runs the code shipped in the `components_repo`. Only use them with repositories you trust, and review their code before passing different ones.

```python
import numpy as np
import torch
from diffusers import ModularPipelineBlocks
from huggingface_hub import hf_hub_download

blocks = ModularPipelineBlocks.from_pretrained(
    "OzzyGT/motionbricks_soma_custom_blocks",
    trust_remote_code=True,
    components_repo="OzzyGT/motionbricks_soma_diffusers",
    trust_components_code=True,
)
pipe = blocks.init_pipeline()
pipe.load_components(dtype=torch.float32)
pipe.to("cuda")

first = hf_hub_download("OzzyGT/diffusers-examples", "motionbricks_soma_diffusers/first.npz", repo_type="dataset")
second = hf_hub_download("OzzyGT/diffusers-examples", "motionbricks_soma_diffusers/second.npz", repo_type="dataset")
context = dict(np.load(first))
target = dict(np.load(second))

motion = pipe(context=context, target=target, target_offset=(1.4, 0.0), num_target_poses=4, output="motion")[0]
np.savez_compressed("transition.npz", **{key: value.cpu().numpy() for key, value in motion.items()})
```

Both clips come from [Kimodo SOMA Modular](https://huggingface.co/OzzyGT/Kimodo_SOMA_Modular): the start of a run, and a standing pose that `target_offset` places 1.4 m ahead of it. The model accepts SOMA30 or SOMA77 poses at 30 FPS and returns a SOMA30 motion; here it brings the run to a stop in 30 frames. See the [custom blocks](https://huggingface.co/OzzyGT/motionbricks_soma_custom_blocks) for the options.

Float32 reproduces NVIDIA's ONNX graph with PyTorch's default settings; float16 halves the memory with small drift, while bfloat16 changes the motion noticeably.

## Licenses

The model weights are licensed under the [NVIDIA Open Model Agreement](LICENSE), the same as the original. The model code in this repository is Apache-2.0. See [NOTICE](NOTICE) for what was changed.