---
language:
- en
license: apache-2.0
base_model:
- nvidia/Kimodo-SOMA-RP-v1.2
library_name: diffusers
tags:
- text-to-motion
- motion-generation
- kimodo
- modular-diffusers
- soma
---

# Kimodo SOMA Modular Diffusers

<video controls loop muted playsinline src="https://huggingface.co/datasets/OzzyGT/diffusers-examples/resolve/main/Kimodo_SOMA_Modular/Kimodo_SOMA_RP_v1_2_run_seed42.mp4"></video>

*A four-second motion generated with the code below, seed `42`, at normal playback speed. The cameras follow the original root motion.*

Modular Diffusers blocks for [NVIDIA Kimodo SOMA RP v1.2](https://huggingface.co/nvidia/Kimodo-SOMA-RP-v1.2): generate human motion from a text prompt. The model code and weights are in [OzzyGT/Kimodo_SOMA_RP_v1_2](https://huggingface.co/OzzyGT/Kimodo_SOMA_RP_v1_2).

The same blocks also support [Kimodo SOMA RP v1.1](https://huggingface.co/OzzyGT/Kimodo_SOMA_RP_v1_1); set `components_repo` to that repository to use the earlier model.

Requires PyTorch, `diffusers==0.41.0`, `transformers==5.20.2`, `peft==0.21.2`, `accelerate>=1.15`, `safetensors>=0.8`, and `numpy`.

## Licensing

The blocks are [Apache-2.0](LICENSE), adapted from [Kimodo](https://github.com/nv-tlabs/kimodo), with [LLM2Vec](https://github.com/McGill-NLP/llm2vec) helpers under [MIT](LICENSE-LLM2VEC). See [NOTICE](NOTICE). This repository contains no model weights. Motion weights use the NVIDIA Open Model License; the text encoder uses the Meta Llama 3 Community License. See the [model repository](https://huggingface.co/OzzyGT/Kimodo_SOMA_RP_v1_2#licenses) for those terms. Built with Meta Llama 3.

## Usage

Generate a four-second motion and save it as NPZ:

```python
import numpy as np
import torch
from diffusers import ModularPipelineBlocks

blocks_path = "OzzyGT/Kimodo_SOMA_Modular"
model_id = "OzzyGT/Kimodo_SOMA_RP_v1_2"

blocks = ModularPipelineBlocks.from_pretrained(
    blocks_path,
    trust_remote_code=True,
    components_repo=model_id,
    trust_components_code=True,
)
pipe = blocks.init_pipeline()
pipe.load_components(dtype={"text_encoder": torch.bfloat16, "default": torch.float32})
pipe.to("cuda")

motion = pipe(
    prompt="A person runs forward at a steady pace.",
    num_frames=120,
    seed=42,
    num_inference_steps=100,
    output="motion",
)
np.savez_compressed(
    "motion.npz",
    **{key: value[0].cpu().numpy() for key, value in motion.items()},
)
```

Output is 30 FPS, meters, +Y up, initial +Z forward, SOMA77, with root motion preserved. Vary `seed` for another take. Negative prompts are not supported.