README.md

---
library_name: motius
pipeline_tag: video-to-video
license: other
tags:
  - human-motion
  - monocular-motion-capture
  - soma-x
---

# GEM-X

GEM-X is NVIDIA's monocular whole-body motion estimator built around the
SOMA-X parametric body model.

- Paper lineage:
  [GEM: A Generalist Model for Human Motion](https://arxiv.org/abs/2505.01425)
- Official source: [NVlabs/GEM-X](https://github.com/NVlabs/GEM-X)
- Motius checkpoint:
  [ZeyuLing/Motius-GEM-X](https://huggingface.co/ZeyuLing/Motius-GEM-X)
- Pinned source revision: `32992550dba114c62243fb55e361311972dce8f9`
- Pinned SOMA-X revision: `e0f8ff0ecfa3edbbb6058b1e0f08822ee2f84ee5`
- Checkpoint SHA-256:
  `4c1f85ca8c1e11e6588aead49fbc024bf660708def670043e0b537c101ee298e`

**Tasks:** Monocular Motion Capture

## Supported Tasks

| Task | Public API | Input | Output |
| --- | --- | --- | --- |
| Monocular Motion Capture | `infer_monocular_motion_capture` | RGB video | `MonocularCaptureResult` |

## Checkpoint

The Motius Hugging Face artifact is complete: GEM-X, SAM-3D-Body, DINOv3,
ViTPose, YOLOX, MHR, SOMA-X identity/corrective assets, and normalization
statistics are stored under their expected paths with SHA-256 provenance.
`Pipeline.from_pretrained` therefore needs no source checkout or second model
download.

## Motion Representation

GEM-X natively predicts SOMA-77:

- 77-joint axis-angle pose;
- 45 identity coefficients;
- 69 global/body-part scale parameters;
- camera and world root translations;
- 77 named joints and 4,505-vertex low-LOD SOMA meshes.

Motius keeps SOMA-X native. It does not manufacture SMPL vertices from a
different topology. Cross-model 3DPW evaluation uses the audited
`common_hmr15_named_v1` joint subset; PVE is reported as unavailable.

## Usage

Create an isolated environment without cloning GEM-X:

```bash
python3.10 -m venv outputs/envs/gem-x
outputs/envs/gem-x/bin/pip install -e ".[gem-x]"
```

Run the standard task API:

```python
from pathlib import Path

from motius.motion.representation.monocular_capture import (
    save_monocular_capture_result,
)
from motius.pipelines.gem_x import GemXPipeline

pipeline = GemXPipeline.from_pretrained(
    "ZeyuLing/Motius-GEM-X",
    bundle_kwargs={
        "python_executable": "outputs/envs/gem-x/bin/python",
    },
)
result = pipeline.infer_monocular_motion_capture(
    "input.mp4",
    output_root="outputs/gem_x/run_001",
    materialize_geometry=True,
    render=True,
)
save_monocular_capture_result(
    result,
    Path("outputs/gem_x/run_001/result.npz"),
)
```

`render=True` requests the upstream keypoint, in-camera, and world previews.
Leave it off for evaluation and batch inference.

## Demo

This 768px, 30 FPS preview renders the world-space SOMA-X mesh returned by the
public Motius pipeline.

[![GEM-X world-motion preview](https://raw.githubusercontent.com/ZeyuLing/Motius/main/assets/model_zoo/gem_x/gem_x_tennis_world.webp)](https://github.com/ZeyuLing/Motius/blob/main/assets/model_zoo/gem_x/gem_x_tennis_world.mp4)

## Evaluation Results

Protocol: `3dpw_test_camera_v1`, all 24 test videos and 37 official person
tracks, evaluated on `common_hmr15_named_v1`.

| Coverage | MPJPE ↓ | PA-MPJPE ↓ | Acceleration ↓ |
| ---: | ---: | ---: | ---: |
| 100.00% | 84.38 mm | 53.20 mm | 5.616 m/s² |

The official demo emits an identity camera trajectory when no external visual
odometry is supplied. These are camera-space metrics; world-space ranking is
unavailable for that run.

## Stage Parity

The strict gate compares all persisted boundaries. Deterministic fields are
bitwise exact. The official contact IK/SOMA CUDA postprocess is non-deterministic
across independent processes, so only its 15 explicitly named descendants use
a hard `3e-6` absolute-error ceiling. No global tolerance is applied.

| Boundary | Fields | Requirement | Result |
| --- | ---: | --- | --- |
| Tracking | 2 | exact | pass |
| Keypoints and camera | 2 | exact | pass |
| Visual features | 2 | exact | pass |
| Complete model input | 18 | exact | pass |
| Raw network output and deterministic decoded fields | 27 | exact | pass |
| Contact-postprocessed model fields | 7 | `atol ≤ 3e-6` | max `6.71e-7` |
| SOMA geometry | 4 | `atol ≤ 3e-6` | max `2.38e-6` |
| Public result | 9 | exact except 4 descendants | max `9.54e-7` |
| **Total** | **71** | **field-scoped policy** | **pass** |

```bash
python tools/verify_monocular_pipeline_parity.py \
  --profile gem-x \
  --reference outputs/parity/gem_x/reference_trace.npz \
  --candidate outputs/parity/gem_x/motius_trace.npz
```

## License

GEM-X source is Apache-2.0 and the public weights use the NVIDIA Open Model
License. SAM-3D-Body, DINOv3, SOMA-X, MHR, and their assets retain their own
terms. See the
[GEM-X attributions](https://github.com/ZeyuLing/Motius/blob/main/motius/models/gem_x/ATTRIBUTIONS.md)
and the packaged third-party notices before use.

## Direct Loading

```python
from motius import Pipeline

pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-GEM-X")
```