docs/source/user_guide/constraints.md

Constraints JSON Format

The --constraints flag in the CLI expects a JSON file containing a list of constraint objects. It is easiest to look at the examples provided with the demo to see how these are formatted. These can be seen for various model types in kimodo/assets/demo/examples.

Tip: the easiest way to get a valid constraints file is to create constraints in the interactive demo and to click on Save Constraints.

High-Level Structure

• The file is a JSON array: [{...}, {...}, ...]
• Each element is an object with at least:
  • type (string)
    • root2d, fullbody, left-hand, right-hand, left-foot, right-foot, end-effector
  • frame_indices (array of integers): 0-based frame indices within the generated clip.

For SOMA models, constraints may be authored or displayed on the full `somaskel77` skeleton, but Kimodo converts them to the reduced `somaskel30` representation before passing them to the model. See the [skeleton](../key_concepts/skeleton.md) section for more details.

Coordinate Space and Units

All spatial values in constraints use the same coordinate system as Kimodo's internal motion representation:

• Axes: Y-up, with locomotion on the XZ ground plane. The Y axis points up, X and Z span the horizontal ground plane.
• Units: Meters. Joint positions, root translations, and 2D root coordinates are all in meters.

Canonicalization

During training, every motion is canonicalized so that the (smoothed) root starts at the XZ origin (0, 0) at frame 0. The initial body heading (facing direction) is randomly rotated and passed to the model as an explicit input (first_heading_angle), so the model is robust to arbitrary initial orientations.

At inference, constraints should be authored relative to this canonical origin:

• smooth_root_2d values at frame 0 should be at (0, 0), with subsequent frames expressing displacement from there.
• root_positions XZ components follow the same convention; Y is the absolute hip height above the ground (typically ~0.9 m for a standing pose, lower for crouching/sitting).
• first_heading_angle (a generation parameter, not part of the constraints JSON) defaults to 0.0 radians (facing +Z) but can be set to any value to change the initial facing direction.

Field-specific notes

Field	Space	Notes
smooth_root_2d	[x, z] ground plane (meters)	Relative to the canonical origin.
root_positions	[x, y, z] (meters)	Y is absolute hip height above ground. XZ relative to canonical origin.
global_root_heading	[cos(θ), sin(θ)]	Not a raw radian value — must be a 2-element cosine/sine pair per frame (i.e. the heading direction vector).
local_joints_rot	axis-angle (radians)	Local joint rotations in the skeleton's rest-pose frame.

Constraints not at frame 0

Adding a constraint at frame 0 is not required. If the first constrained frame is later in the sequence (e.g. frame 45), Kimodo generates the initial frames freely from its learned distribution, starting near XZ = (0, 0) with the heading set by first_heading_angle. The constraint just needs to be reachable from that starting configuration given the text prompt and motion duration.

Constraint Types

Depending on type, additional fields are required or optional. All numeric arrays are plain nested JSON lists. In the following definitions T is the number of constrainted frames (i.e., number of frame_indices) and J is the number of skeleton joints.

root2d

This captures 2D root waypoints and 2D root paths. It requires:

• smooth_root_2d (array shapes [T, 2]): Smoothed root positions [x, z] on the ground plane at the given frame_indices.

and optionally:

• global_root_heading (array shapes [T, 2]): Global root heading direction [cos, sin] at the given frame_indices.

fullbody

This captures full-body keyframe constraints on joint positions. It includes:

• local_joints_rot (array shaped [T, J, 3]): Per-frame per-joint axis-angle local rotations (radians). Constraint joint positions will be derived from these.
• root_positions (array shaped [T, 3]): Root (hips) translation [x, y, z].
• smooth_root_2d (optional; array of [T, 2]): Smoothed root positions [x, z]. If omitted, it is taken as the [x, z] components of root_positions.

Note the local_joint_rot will not explicitly be constrained, the constraint will be on the joint positions that results from FK with the given joint rotations.

left-hand / right-hand / left-foot / right-foot

Captures end-effector constraints on the hand/feet joint positions and global rotations.

These use the same fields as fullbody. However, under the hood these will only affect the corresponding end-effectors and hips. Each of these types is a shorthand for end-effector with pre-set joint names.

end-effector

A general end-effector constraint that requires an additional field:

• joint_names (array of strings): Which end-effectors to constrain (e.g. ["left_hand"], ["right_foot", "left_foot"]). Available names depend on the skeleton; see the skeleton's expand_joint_names() for the full mapping.

Otherwise uses the same fields as fullbody (local_joints_rot, root_positions, optional smooth_root_2d).

Examples

Root 2D waypoints

[
  {
    "type": "root2d",
    "frame_indices": [0, 30, 60],
    "smooth_root_2d": [[0.0, 0.0], [0.5, 0.0], [1.0, 0.1]]
  }
]

Full-body keyframe

[
  {
    "type": "fullbody",
    "frame_indices": [60],
    "root_positions": [[0.0, 0.96, 1.5]],
    "local_joints_rot": [[[0.0, 0.0, 0.0], "... one [3] per joint ..."]]
  }
]

Here root_positions places the hips at x=0, y=0.96 m (standing height), z=1.5 m forward from the origin. local_joints_rot is a [T, J, 3] array of axis-angle rotations for every joint in the skeleton.