OzzyGT/Kimodo_SOMA_Modular

Kimodo SOMA Modular Diffusers

A four-second motion generated with the code below, seed 42, at normal playback speed. The cameras follow the original root motion.

Modular Diffusers blocks for NVIDIA Kimodo SOMA RP v1.2: generate human motion from a text prompt. The model code and weights are in OzzyGT/Kimodo_SOMA_RP_v1_2.

The same blocks also support Kimodo SOMA RP v1.1; set components_repo to that repository to use the earlier model.

Requires PyTorch, diffusers==0.41.0, transformers==5.1.0, peft==0.21.2, accelerate>=1.15, safetensors>=0.8, and numpy.

Licensing

The blocks are Apache-2.0, adapted from Kimodo, with LLM2Vec helpers under MIT. See NOTICE. This repository contains no model weights. Motion weights use the NVIDIA Open Model License; the text encoder uses the Meta Llama 3 Community License. See the model repository for those terms. Built with Meta Llama 3.

Usage

Generate a four-second motion and save it as NPZ:

import numpy as np
import torch
from diffusers import ModularPipelineBlocks

blocks_path = "OzzyGT/Kimodo_SOMA_Modular"
model_id = "OzzyGT/Kimodo_SOMA_RP_v1_2"

blocks = ModularPipelineBlocks.from_pretrained(
    blocks_path,
    trust_remote_code=True,
    components_repo=model_id,
    trust_components_code=True,
)
pipe = blocks.init_pipeline()
pipe.load_components(dtype={"text_encoder": torch.bfloat16, "default": torch.float32})
pipe.to("cuda")

motion = pipe(
    prompt="A person runs forward at a steady pace.",
    num_frames=120,
    seed=42,
    num_inference_steps=100,
    foot_cleanup=False,
    output="motion",
)
np.savez_compressed(
    "motion.npz",
    **{key: value[0].cpu().numpy() for key, value in motion.items()},
)

Output is 30 FPS, meters, +Y up, initial +Z forward, SOMA77, with root motion preserved. Vary seed for another take. Negative prompts are not supported. Optional foot_cleanup=True requires the Kimodo geometry utilities.