Add MotionBricks model release files

README.md

@@ -0,0 +1,82 @@
+---
+library_name: pytorch
+tags:
+  - nvidia
+  - motionbricks
+  - motion-generation
+  - pose-conditioned
+  - robotics
+  - animation
+---
+
+# MotionBricks
+
+[Project page](https://nvlabs.github.io/motionbricks/) · [Paper](https://arxiv.org/abs/2604.24833)
+
+MotionBricks generates motion in real time for humanoid robots and human characters.
+Give it the character's recent poses and the poses it should reach next, and it
+generates the movement connecting them.
+
+Smart Primitives turn target poses into reusable building blocks for navigation
+and object interactions, without retraining the model. The accompanying software
+includes SOMA and Unitree G1 demos built with Newton and Warp, plus tools for
+creating your own primitives and scenes.
+
+## Models
+
+This repository contains pretrained SafeTensors weights for two model families:
+
+| Model | Use | Details |
+| --- | --- | --- |
+| SOMA | Generate human motion on the SOMA skeleton. | [Model card](soma/README.md) |
+| Unitree G1 | Generate motion for the Unitree G1 humanoid. | [Model card](g1/README.md) |
+
+Each folder includes the model's weights, configuration and supporting files.
+Keep them together. The SOMA and G1 files are not interchangeable.
+
+## Get started
+
+**To try the demos**, start with the [MotionBricks software](https://github.com/NVlabs/motionbricks).
+It includes ready-to-use ONNX models, so you do not need to download these
+checkpoints to run it. The TensorRT demo runtime requires an NVIDIA GPU.
+
+**To export ONNX models or build another runtime**, download the weight packages:
+
+```sh
+hf download nvidia/MotionBricks --local-dir MotionBricks
+```
+
+For just one model, add `--include "soma/*"` or `--include "g1/*"`.
+The files are already unpacked. Follow the software's `tools/onnx` guide to export
+them, using the matching family folder.
+
+## Model details and limitations
+
+The family cards explain each model's inputs, outputs, training data, evaluation
+and limitations. Each model uses three networks together. These weight
+packages are not Transformers `from_pretrained` models, and this repository does
+not include a training pipeline or the full training data. The packages contain
+weights and inference metadata, not the optimizer and training-loop state needed
+to resume the original training runs.
+
+The G1 package does not include the original demo motion bank, `out/G1-clip.ckpt`.
+Demo motion examples are separate from the model weights.
+
+MotionBricks generates motion, not a controller for a physical robot. Running the
+motion on hardware requires a separate tracking controller, robot-limit checks
+and safety validation.
+
+## File verification
+
+Each model folder includes a file manifest and SHA-256 checksums:
+
+- SOMA: [manifest](soma/manifest.json) · [checksums](https://huggingface.co/nvidia/MotionBricks/blob/main/soma/SHA256SUMS)
+- Unitree G1: [manifest](g1/manifest.json) · [checksums](https://huggingface.co/nvidia/MotionBricks/blob/main/g1/SHA256SUMS)
+
+## Terms
+
+- **Models:** the SOMA and G1 weights use the [NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/). It permits commercial use and redistribution of modified models, subject to its terms.
+- **Software:** the accompanying MotionBricks code uses [Apache 2.0](https://github.com/NVlabs/motionbricks/blob/main/LICENSE).
+- **Motion samples:** the separately bundled BONES samples and listed derivatives use the [NVIDIA Sample Data Evaluation License](https://github.com/NVlabs/motionbricks/blob/main/assets/motions/LICENSE.txt), for internal evaluation and testing of NVIDIA technologies. See the [sample notice](https://github.com/NVlabs/motionbricks/blob/main/assets/motions/README.md) for the exact files covered.
+
+The model agreement does not replace the software, sample-data or third-party asset terms. This repository does not include the sample motions or the training dataset.

SHA256SUMS

@@ -0,0 +1,61 @@
+11ad7efa24975ee4b0c3c3a38ed18737f0658a5f75a0a96787b576a78a023361  .gitattributes
+5e7c69d495522d39130973748a07e62af9fe294914ce46de56a002d2c16818ee  README.md
+adcb83b61946ffe4ac974cc3d2e7c2aa8f9832dbe75e0c29e16d53b0ae0aa44c  g1/README.md
+d4d0a026810ce063157afa5609f90825adbc8c5e717525c8dfb53721e9a3d731  g1/SHA256SUMS
+d8bd82393b308d27bfc3cb1670d4e4723794c3b349fa3b83435ce0f93e5b5065  g1/manifest.json
+359a134cce0e863346024bc8fe37286666e2e35c98588267ef4bb989bd67280d  g1/out/SHA256SUMS
+5f4bfa948eadb13a7b0d186ed2cba2e6f837e6709898b8e932aa345ff853515e  g1/out/motionbricks_pose/version_1/checkpoints/model-step=2000000.json
+2154d80cac2beb4bb4074b4db92ae59d9fdfbc58651550c32941fe8527d28313  g1/out/motionbricks_pose/version_1/checkpoints/model-step=2000000.safetensors
+273af770d328b458510ee6049fac8e38fa486c8792cc27b3515dddc094a7cd1f  g1/out/motionbricks_pose/version_1/config.yaml
+3c482b9f810a9c4921d2e9fe46dae4e9e70d48ac9ea7d7b19a91fad2490d75e5  g1/out/motionbricks_pose/version_1/hparams.yaml
+f70ba7887c5db6cd77739e573bbb734af0ec79a8cbabd9e435b7c72a5d040663  g1/out/motionbricks_pose/version_1/meta.yaml
+6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0  g1/out/motionbricks_pose/version_1/skeleton/joints.npy
+02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4  g1/out/motionbricks_pose/version_1/skeleton/parents.npy
+5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050  g1/out/motionbricks_pose/version_1/skeleton/skeleton.yaml
+ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea  g1/out/motionbricks_pose/version_1/stats/motion/mean.npy
+fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7  g1/out/motionbricks_pose/version_1/stats/motion/std.npy
+5df3e2521ba80997251c72bf2f5dd22cea301df458a407ab34d5e75f28855b5e  g1/out/motionbricks_root/version_1/checkpoints/model-step=2000000.json
+da6ff2c05e45dfe4ac5e8e8362a6404f5c927fcddb18410e6804b61e55d4f2dc  g1/out/motionbricks_root/version_1/checkpoints/model-step=2000000.safetensors
+b174f03a333f7f7857c3e2b8a32da517caddd198f228efeea2e203d046ce212a  g1/out/motionbricks_root/version_1/config.yaml
+15dfadeab65b8ab1931b4e5257a2f9ebfa57e559510af1bd5baa96cfe2baeaf5  g1/out/motionbricks_root/version_1/hparams.yaml
+c541e9d63b9b876a56c031a360d4d3929013b83515510a10f47ae63f6c7c8f17  g1/out/motionbricks_root/version_1/meta.yaml
+6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0  g1/out/motionbricks_root/version_1/skeleton/joints.npy
+02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4  g1/out/motionbricks_root/version_1/skeleton/parents.npy
+5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050  g1/out/motionbricks_root/version_1/skeleton/skeleton.yaml
+ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea  g1/out/motionbricks_root/version_1/stats/motion/mean.npy
+fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7  g1/out/motionbricks_root/version_1/stats/motion/std.npy
+ceaa8820137c55ddc472a499307e8fbf4f9e9e4c503e73b967fa43ae0afc2141  g1/out/motionbricks_vqvae/version_1/checkpoints/model-step=2000000.json
+bf9c31173b334303bf392d184c24701ad1a095b3e94a5fe6b11db4d01343a454  g1/out/motionbricks_vqvae/version_1/checkpoints/model-step=2000000.safetensors
+027a2d7ba5f49cadeadbcc9a6b0c6784d657d5a842e41ff820c5233f1cd6c1f3  g1/out/motionbricks_vqvae/version_1/config.yaml
+7af860d6c144ff71ccdb4085b8413be892194ef419031b81e231502c2181576d  g1/out/motionbricks_vqvae/version_1/hparams.yaml
+7bdf23605dcb89708a39bae33e12de59ebf4d216ba09b91931a2f813cc57e8c7  g1/out/motionbricks_vqvae/version_1/meta.yaml
+6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0  g1/out/motionbricks_vqvae/version_1/skeleton/joints.npy
+02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4  g1/out/motionbricks_vqvae/version_1/skeleton/parents.npy
+5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050  g1/out/motionbricks_vqvae/version_1/skeleton/skeleton.yaml
+ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea  g1/out/motionbricks_vqvae/version_1/stats/motion/mean.npy
+fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7  g1/out/motionbricks_vqvae/version_1/stats/motion/std.npy
+50ac7be9443f1bd677a648c610f29b8dd9c6917dddc95a89f7bdbad410638744  soma/README.md
+a740044923f5dd020d9bdfe2e7a11b181dfab16fae9d0091e57850e082cb964d  soma/SHA256SUMS
+9ae6cd2d4386b5178e62d47f03f7a1750326a729b919f545fe5cd4a4b966bff0  soma/manifest.json
+d7ead5ef043a9d1e99071b6f8aedc70753560fdc0a613cc1287f1561b50fce3c  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/SHA256SUMS
+4ad901e6eb008f40bb5eae22edf25a9d42b3ac553e0c55657a63a0e218d8d015  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json
+8537d9d33a880ff1df6c16c28ed061d60c461a5918d70b1cf99251ff91a10a10  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors
+c2e3b54d5c6aaf0fbcd1e57b61c9306991bdaf841a609c088944c4a460012885  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml
+57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy
+8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml
+802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy
+1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy
+70571a550fda3cfa9fdb1bd3fed502dde27e1f3bcac58546af54a4e8e14705a3  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json
+a48d48c3d8ff98c825ca315797644789ae37f2db9069779a3118f42e84f4f29e  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors
+6ef85c1296e5ba860f79920222684383aa76573b9dbf3299378f05eaf7971d56  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml
+57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy
+8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml
+802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy
+1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy
+f3518cde99c51f5177a08c1226a45d9992d23e9bb985e658847957cc287a9d00  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json
+ba0d1db48ba7182ef5d1a72268bfaa1a196f1f1fbf00144c046d947b90684a1a  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors
+826e58115a0baef638f5a4e42f477d12776a82ac6f0dcda16bc6b8454a88342e  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml
+57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy
+8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml
+802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy
+1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6  soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy

g1/README.md

@@ -0,0 +1,453 @@
+---
+library_name: pytorch
+tags:
+  - nvidia
+  - motionbricks
+  - unitree-g1
+  - motion-generation
+  - pose-conditioned
+  - robotics
+---
+
+# MotionBricks Unitree G1
+
+[Project page](https://nvlabs.github.io/motionbricks/) · [Paper](https://arxiv.org/abs/2604.24833) · [Original G1 implementation](https://github.com/NVlabs/GR00T-WholeBodyControl/tree/main/motionbricks)
+
+## Model Description
+
+MotionBricks Unitree G1 generates robot reference motion from recent motion, target poses,
+and root-motion constraints. Applications can update targets and feed generated motion back
+into the model to produce continuous locomotion and transitions. A separate controller is
+needed to execute that motion on a physical robot.
+
+NVIDIA developed the original pretrained weights for the MotionBricks
+subproject in GR00T Whole-Body Control. This package stores those weights in SafeTensors format.
+The model uses a 34-joint skeletal representation of Unitree G1, with separate components for
+pose tokenization, pose prediction, and root-motion prediction. Its pose tokenizer is a
+vector-quantized variational autoencoder (VQ-VAE).
+
+This model is intended for commercial and non-commercial motion-generation applications.
+
+### Where to Start
+
+To run MotionBricks in an application, start with the [G1 ONNX model and runtime](https://github.com/NVlabs/motionbricks/blob/main/docs/unitree_g1_model.md).
+Use the checkpoints on this page to work with the individual PyTorch components or create
+ONNX exports. [Download the complete G1 package](#download-and-files), then follow the
+[exporter guide](https://github.com/NVlabs/motionbricks/blob/main/tools/onnx/README.md). A training pipeline is not included.
+
+| Property | Value |
+| --- | --- |
+| Model family | MotionBricks |
+| Model version | MotionBricks-G1-RP-v1 |
+| Task | Pose-conditioned robot motion generation and inbetweening |
+| Robot | Unitree G1, floating pelvis and 29 joint angles |
+| Skeleton | Unitree G1, 34 model joints |
+| FPS | 30 |
+| Checkpoint components | Pose VQ-VAE, pose model, root model |
+| Checkpoint step | 2,000,000 for each model |
+| Weight format | SafeTensors (`.safetensors`) |
+
+## License and Terms of Use
+
+Use of this model is governed by the [NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/).
+It permits commercial use and redistribution of modified models, subject to the agreement's terms.
+
+- The accompanying software uses [Apache 2.0](https://github.com/NVlabs/motionbricks/blob/main/LICENSE).
+- The listed Bones-derived human BVH and G1 CSV samples use the
+  [NVIDIA Sample Data Evaluation License](https://github.com/NVlabs/motionbricks/blob/main/assets/motions/LICENSE.txt).
+  They are for internal evaluation and testing of NVIDIA technologies. See the
+  [sample notice](https://github.com/NVlabs/motionbricks/blob/main/assets/motions/README.md) for the exact files covered.
+- Robot assets and third-party dependencies retain their [respective licenses](https://github.com/NVlabs/motionbricks/tree/main/licenses).
+
+## Deployment Geography
+
+Global.
+
+## Intended Uses
+
+**Use Case:** Pose-conditioned G1 reference-motion generation for animation, robotics simulation,
+and downstream control experiments.
+
+**Intended Users:** Developers and researchers integrating G1 motion generation into their applications.
+Generated motion is not validated for physical robot control.
+
+## Model Package
+
+This card covers `MotionBricks-G1-RP-v1` in the
+[`g1/` folder](https://huggingface.co/nvidia/MotionBricks/tree/main/g1).
+The SOMA package is documented separately.
+
+The weights come from the original
+[GR00T Whole-Body Control implementation](https://github.com/NVlabs/GR00T-WholeBodyControl/tree/b042411fae38ee4d1af9aac82a37a1f8d14d6dd0/motionbricks/out).
+They are stored in SafeTensors format with numeric skeleton arrays. The accompanying Open Neural Network Exchange (ONNX)
+export changes the runtime interface, not the training of these weights.
+
+## References
+
+T. Wang, O. Dionne, M. De Ruyter, D. Minor, D. Rempe, K. Zhao, M. Petrovich,
+Y. Yuan, C. Li, Z. Luo, B. Robison, X. Blackwell, B. Antoniazzi, X. B. Peng,
+Y. Zhu, and S. Yuen, **MotionBricks: Scalable Real-Time Motions with
+Modular Latent Generative Model and Smart Primitives**, 2026.
+[ACM Transactions on Graphics](https://doi.org/10.1145/3811334). [arXiv:2604.24833](https://arxiv.org/abs/2604.24833).
+
+## Download and Files
+
+Download the G1 model files from the shared MotionBricks repository:
+
+```sh
+hf download nvidia/MotionBricks --include "g1/*" --local-dir MotionBricks
+```
+
+The expanded files preserve the original `out/` layout under `g1/`. No archive
+extraction is required.
+
+| Component | Location within the `g1/` folder |
+| --- | --- |
+| Pose VQ-VAE | `out/motionbricks_vqvae/version_1/` |
+| Pose model | `out/motionbricks_pose/version_1/` |
+| Root model | `out/motionbricks_root/version_1/` |
+
+Each model directory contains `checkpoints/model-step=2000000.safetensors`, a
+JSON metadata sidecar, saved configuration, numeric normalization arrays, and
+skeleton metadata. Skeleton arrays use `joints.npy` and `parents.npy`. Keep these
+files together. The original demo motion bank, `out/G1-clip.ckpt`, is not included.
+It is not one of the three neural-network checkpoints. The original demo requires that
+separate motion bank.
+
+[manifest.json](https://huggingface.co/nvidia/MotionBricks/blob/main/g1/manifest.json) records each included file's size and SHA-256.
+[SHA256SUMS](https://huggingface.co/nvidia/MotionBricks/blob/main/g1/SHA256SUMS) provides the checksums for the expanded model files.
+
+## Model Architecture
+
+**Architecture Type:** Modular latent generative model.
+
+**Network Architecture:** Transformer-based pose and root predictors with a learned pose
+tokenizer and decoder. Recent motion, target poses, root constraints, and allowed duration
+condition generation. The pose predictor produces motion codes, which the decoder turns into
+skeletal-motion features. The root predictor generates root motion and duration.
+
+**Base Model:** The original G1 components were trained from scratch on RP1 motion retargeted
+to G1. This release reuses those weights without retraining and packages them in SafeTensors
+format.
+
+The model operates on the G1 skeletal representation. Application code converts between
+skeletal motion and the robot's joint angles.
+
+The original G1 tokenizer uses eight heads with 10 codes per head and 32
+dimensions per head. Use the matching G1 bundle and configuration for
+reproduction.
+
+**Number of Model Parameters:** Approximately 195 million across the three components. The
+component counts below are rounded.
+
+| Component | Approximate parameters |
+| --- | --- |
+| Pose VQ-VAE | 23.8 million |
+| Pose predictor | 136.6 million |
+| Root predictor | 34.1 million |
+
+## Inputs
+
+**Input Type(s):** Skeletal-motion features, pose constraints, root-motion constraints, motion-token
+IDs, conditioning masks, and duration constraints. Text, images, audio, and video are not inputs
+to these checkpoint components.
+
+**Input Format(s):** Numeric PyTorch tensors. Motion features use 32-bit floating point (FP32) with the bundled
+normalization statistics. Token IDs and lengths are integers. Constraint and padding masks are
+Boolean.
+
+**Input Parameters:** The component tables below specify the tensor shapes and meanings.
+
+**Other Properties Related to Input:** Context and target poses supply constraints on a short
+generation. The root predictor receives four context poses and four target poses. All components
+require their matching configuration and normalization statistics.
+
+The three checkpoint components use the tensor interfaces below. `N` is the
+batch size, `L` is the number of motion-token positions, and `T = 4L` is the
+number of frames. Generation uses 6 to 16 tokens (24 to 64 frames) at 30 Hz.
+Shorter sequences can be padded and accompanied by their valid lengths.
+The checkpoint networks support batching. The practical batch size depends
+on memory and the inference implementation.
+
+These are component-level interfaces after feature preparation. An application's pose
+interface or exported graph can differ.
+
+For joint ordering, coordinate conventions, normalization, and motion
+reconstruction, use the matching MotionBricks implementation and the bundled
+configuration and skeleton metadata.
+
+### Pose VQ-VAE: tokenizer and decoder
+
+| Tensor | Direction | Shape | Description |
+| --- | --- | --- | --- |
+| Pose features | Encoder input | `(N, T, 304)` | Root height, 33 non-root joint positions (3 values each), and 34 joint rotations (6 values each) |
+| Motion codes | Encoder output / decoder input | `(N, L, 8)` | Eight codebook heads. Each ID is in `0…9` |
+| Target pose features | Decoder input | `(N, T, 304)` | Pose constraints in the same feature representation |
+| Target pose mask | Decoder input | `(N, T)` | Which frames contain pose constraints |
+| Local root velocity | Decoder input | `(N, T, 2)` | Ground root-motion conditioning |
+| Valid-token mask | Decoder input, optional | `(N, L)` | Valid positions when decoding padded sequences |
+| Reconstructed motion | Decoder output | `(N, T, 413)` | Local-root features (4), joint positions (99), 6D rotations (204), joint velocities (102), and foot-contact features (4) |
+
+Each token represents four frames. The quantized latent has 256 values per
+token (eight heads × 32 values). The table uses separate per-head code IDs.
+The implementation also supports packing them into one integer per token.
+
+### Pose predictor
+
+| Tensor | Direction | Shape | Description |
+| --- | --- | --- | --- |
+| Motion tokens | Input | `(N, L, 8)` | Code IDs `0…9`, or mask ID `10` for unknown codes |
+| Root trajectory | Input | `(N, T, 4)` | Ground root position and two-component heading, without root height |
+| Pose constraints | Input | `(N, T, 304)` | Root height, joint positions, and 6D rotations |
+| Pose constraint mask | Input | `(N, T)` | Frames with supplied pose constraints |
+| Valid token count | Input | `(N, 1)` | One length in `6…16` per sample |
+| Pose-code logits | Output | `(N, L, 8, 10)` | Scores over each head's codebook. Selected codes feed the decoder |
+
+### Root predictor
+
+The root predictor receives eight boundary frames: four context frames and
+four target frames. Each sample can have its own constraints and duration.
+
+| Tensor | Direction | Shape | Description |
+| --- | --- | --- | --- |
+| Global root constraints | Input | `(N, 8, 5)` | Root position (3) and heading (2) |
+| Local root constraints | Input | `(N, 8, 4)` | Root angular velocity (1), ground velocity (2), and height (1) |
+| Pose constraints | Input | `(N, 8, 304)` | Root height, joint positions, and 6D rotations |
+| Constraint masks | Input | Three tensors of `(N, 8)` | One mask for each of the preceding constraint tensors |
+| Requested token count | Input | `(N, 1)` | A specified duration in `6…16`, or the unknown-duration token `18` |
+| Allowed durations | Input, optional | `(N, 11)` | Flags for the allowed token counts `6…16` |
+| Duration logits | Output | `(N, 12)` | Eleven duration scores plus an out-of-reach class |
+| Predicted token count | Output | `(N,)` | Selected duration. Generation disables the out-of-reach class |
+| Predicted global root | Output | `(N, 64, 5)` | Padded root trajectory. Each sample's valid length is four times its token count |
+
+## Outputs
+
+**Output Type(s):** Motion codes and logits, reconstructed skeletal-motion features, root-motion
+predictions, and valid duration.
+
+**Output Format(s):** Numeric PyTorch tensors. Motion features and root predictions use FP32.
+Motion codes and selected lengths are integers.
+
+**Output Parameters:** The component tables above specify each output's shape and meaning.
+Output motion is represented at 30 Hz. Valid-length information distinguishes generated motion
+from padding.
+
+**Other Properties Related to Output:** Longer motion requires repeated generation. Generated
+motion can supply the next context. The checkpoint-component interfaces differ from the joint
+coordinate interface of the combined ONNX export.
+
+Together, the components generate root and body motion features. After
+denormalization and skeletal reconstruction, motion can be represented as
+root translations `(N, T, 3)` and joint rotation matrices
+`(N, T, 34, 3, 3)`. The matching kinematic conversion can derive the robot's
+29 joint angles.
+
+## Software Integration
+
+**Runtime Engine(s):** PyTorch with the matching MotionBricks implementation, configuration,
+normalization statistics, and skeleton metadata. The exporter requires Python 3.12,
+PyTorch 2.14.0+cpu, NumPy 1.26.4, ONNX 1.23.2, and SafeTensors 0.8.0.
+
+**Supported Hardware Microarchitecture Compatibility:** The exporter runs on the CPU.
+A specific CPU model, minimum-memory requirement, and broader compatibility matrix are
+not documented. For ONNX runtime requirements, see the [G1 ONNX card](https://github.com/NVlabs/motionbricks/blob/main/docs/unitree_g1_model.md).
+
+**Supported Operating System(s):** We tested checkpoint-to-ONNX conversion on Windows.
+Tests on other operating systems are not reported.
+
+The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
+
+The [exporter](https://github.com/NVlabs/motionbricks/blob/main/tools/onnx/README.md) accepts the downloaded checkpoint folder.
+Its loader checks required file sizes and SHA-256 hashes, reads SafeTensors weights,
+and disables NumPy pickle loading. Keep the checkpoint and exporter versions matched.
+Do not load untrusted checkpoints. These checks do not replace security scans or
+validation in your application.
+
+The [original implementation](https://github.com/NVlabs/GR00T-WholeBodyControl/tree/main/motionbricks)
+provides model lineage and the earlier checkpoint-based demo. Its original PyTorch checkpoint
+loader is not directly compatible with this SafeTensors package.
+
+The package omits optimizer, scheduler, callback, and training-loop state. It provides weights
+for inference or fine-tuning initialization with a compatible implementation, not exact
+resumption of the original training run.
+
+## Release Date
+
+Hugging Face 10/23/2026 via the [MotionBricks G1 checkpoint package](https://huggingface.co/nvidia/MotionBricks/tree/main/g1).
+
+## Model Versions
+
+`MotionBricks-G1-RP-v1` contains the three original G1 components at training step 2,000,000.
+The [manifest](https://huggingface.co/nvidia/MotionBricks/blob/main/g1/manifest.json) and
+[checksums](https://huggingface.co/nvidia/MotionBricks/blob/main/g1/SHA256SUMS) identify the package files.
+The accompanying software release is v0.2.0.
+
+The components are used together. The MotionBricks exporter combines them into FP32 or
+mixed 16-bit floating-point (FP16) ONNX models. Those models have a [separate card](https://github.com/NVlabs/motionbricks/blob/main/docs/unitree_g1_model.md) and a
+joint-coordinate interface. The export reuses the existing weights without retraining.
+It is not a newly trained joint-angle model.
+
+## Training, Testing and Evaluation Datasets
+
+### Training Dataset
+
+**Name/Identifier:** Bones RigPlay 1 (RP1).
+
+**Data Modality:** Other: human motion capture retargeted to the 34-joint Unitree G1
+skeleton at 30 Hz. The model consumes motion features without text embeddings.
+
+**Training Data Size:** 313,526 selected training clips. The approximately 700 hours describe
+the source corpus, not the selected training subset.
+
+**Data Collection Method:** Automatic/Sensors. Bones captured human performances in a studio and
+supplied skeletal motion. NVIDIA used [SOMA Retargeter](https://github.com/NVIDIA/soma-retargeter)
+to retarget that motion to G1, then converted it into model features. This preparation is not
+direct robot recording.
+
+**Labeling Method:** Automatic/Sensors for captured skeletal motion, followed by automated
+retargeting and feature preparation. Text annotations are not model conditioning
+inputs in the supplied motion-only configuration.
+
+**Properties:** G1 skeletal motion with a capture T-pose and dual-root
+representation, derived from a source corpus of approximately 350,000 RP1
+clips and 700 hours of motion.
+
+**Linguistic Characteristics:** Not applicable to the motion-feature inputs. Associated text
+descriptions are dataset metadata, not model conditioning inputs.
+
+### Testing Dataset
+
+**Name, Size and Role:** We used 313,526 clips for training and a separate, selected set of
+4,881 clips for validation and evaluation during training. No clips are shared between these
+sets. The 4,881 clips are a selection from the held-out pool, not the entire pool. A separate
+untouched final-test set, a performer-disjoint split, and the selected subset's duration are
+not documented.
+
+**Data Collection Method:** Automatic/Sensors, from the same Bones studio motion-capture source
+as the training set. NVIDIA retargeted the human motion to the G1 representation.
+
+**Labeling Method:** Automatic/Sensors for captured skeletal motion, followed by automated
+retargeting and feature preparation.
+
+**Properties:** The validation inputs are 30 Hz skeletal-motion features on the 34-joint G1
+representation. Text descriptions are selection metadata, not conditioning inputs. Linguistic
+characteristics are not applicable to these motion-only inputs. Disjoint clips alone do not rule out other forms of data leakage.
+
+### Evaluation Dataset and Results
+
+**Name/Size:** Evaluation uses the same 4,881-clip RP1 subset described under
+[Testing Dataset](#testing-dataset). It is not an additional independent test set.
+
+**Benchmark Score:** A complete quantitative evaluation report for these checkpoint files
+is not included with this package.
+
+**Data Collection Method:** Automatic/Sensors, using the same studio-captured human motion
+retargeted to G1.
+
+**Labeling Method:** Automatic/Sensors for captured skeletal motion, followed by automated
+retargeting and feature preparation.
+
+**Properties:** Evaluation uses the same 30 Hz G1 skeletal-motion representation as training.
+Text descriptions are selection metadata, not conditioning inputs. Linguistic characteristics
+are not applicable to the motion-only inputs. The subset's duration is unspecified.
+
+**Recorded Results:** The package includes training-step metadata and tested export support,
+but no complete checkpoint-specific quality score table.
+
+| Check | Available evidence |
+| --- | --- |
+| Training progress | All three checkpoints record global step 2,000,000. This is training metadata, not a quality or safety score. |
+| Checkpoint conversion | Checkpoint-to-ONNX conversion tested on Windows with CPU PyTorch. See [Inference](#inference). |
+| Method evaluation | The [MotionBricks paper](https://arxiv.org/abs/2604.24833) reports experiments with its specified configurations, including separately trained benchmark models. |
+
+The paper's benchmark dataset sizes and scores are not measurements of this packaged G1 application.
+
+## Inference
+
+**Acceleration Engine:** The checkpoint implementation and exporter use PyTorch.
+The exporter runs on the CPU. The separate ONNX runtime uses TensorRT 10.8.
+
+**Test Hardware Set-Up:** We tested checkpoint-to-ONNX conversion on Windows using CPU PyTorch.
+The CPU model, conversion time, and checkpoint-runtime latency and throughput are not reported.
+The [ONNX card](https://github.com/NVlabs/motionbricks/blob/main/docs/unitree_g1_model.md) describes the separate Windows and RTX 4090 runtime.
+Neither successful export nor runtime execution establishes safe physical robot tracking.
+
+## Limitations
+
+- The model is specific to the bundled G1 kinematics and hinge order.
+- The generated poses do not enforce dynamic balance, actuator limits,
+  collision avoidance, or physical contact tracking.
+- Physical execution needs a downstream controller and application-specific
+  validation.
+- Generation uses a short sequence of context and target poses. Longer motion requires
+  repeated generation.
+- Numerical precision and backend differences can accumulate through feedback.
+- Scene geometry and text prompts are outside the model's conditioning interface.
+
+## Ethical Considerations
+
+NVIDIA believes Trustworthy AI is a shared responsibility and we have
+established policies and practices to enable development for a wide array of
+AI applications. Developers should work with their internal model team to
+ensure this model meets requirements for the relevant industry and use case
+and addresses unforeseen product misuse.
+
+For more detailed information, see the [Bias](#bias-subcard),
+[Explainability](#explainability-subcard), [Privacy](#privacy-subcard) and
+[Safety & Security](#safety-and-security-subcard) subcards below.
+
+Please report model quality, risk, security vulnerabilities or NVIDIA AI
+Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
+
+## Bias Subcard
+
+| Field | Response |
+| --- | --- |
+| Participation considerations from adversely impacted groups in model design and testing | Not reported for these original G1 checkpoints. |
+| Measures taken to mitigate unwanted bias | A specific measured bias mitigation is not reported. Applications should evaluate the motion conditions and users relevant to their use case. |
+| Bias Metric, if measured | Demographic subgroup and motion-category comparisons are not reported for this checkpoint package. |
+
+Retargeting motion to one robot skeleton does not establish equal performance
+across source performers, motion categories or target constraints.
+
+## Explainability Subcard
+
+| Field | Response |
+| --- | --- |
+| Intended Task/Domain | Pose-conditioned G1 motion generation and inbetweening for robotics simulation, animation previews and control experiments. |
+| Model Type | Modular latent generative model with transformer predictors and a learned motion tokenizer and decoder. |
+| Intended Users | Developers and researchers integrating G1 kinematic motion generation. |
+| Output | Motion codes and logits, root predictions, duration and reconstructed skeletal-motion features, as specified in the component tensor tables. |
+| Describe how the model works | Recent motion and target constraints condition root and pose predictors. The learned tokenizer represents motion with discrete codes, and the decoder reconstructs motion features. Matching application code converts the 34-joint representation to G1 root transforms and 29 joint angles. |
+| Groups tested to produce comparable outcomes | Demographic subgroup comparisons are not reported. |
+| Technical Limitations and Mitigation | Fixed G1 kinematics, short context, no scene-geometry input and backend/precision variation. Validate constraints, motion quality, limits, contacts and any physical execution separately. |
+| Verified to have met prescribed NVIDIA quality standards | Yes |
+| Performance Metrics | See [Evaluation Dataset and Results](#evaluation-dataset-and-results) for evaluation evidence and [Inference](#inference) for runtime evidence. |
+| Potential Known Risks | Invalid or implausible poses, accumulated feedback artifacts and unsafe physical execution if generated references are treated as validated robot commands. |
+| Licensing | Use of this model is governed by the [NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/). See [License and Terms of Use](#license-and-terms-of-use) for separate software and sample-data terms. |
+
+## Privacy Subcard
+
+Training used RP1 skeletal motion collected by Bones and supplied to NVIDIA under a negotiated data license. NVIDIA retargeted the motion to Unitree G1.
+
+The datasets used to train this model were reviewed before release. Training used motion features, not text annotations or interactions with this model.
+
+The package does not include the training corpus. Local execution does not eliminate privacy risks associated with training data.
+
+Please review [NVIDIA’s Privacy Policy](https://www.nvidia.com/en-us/about-nvidia/privacy-policy/) for more information.
+
+## Safety and Security Subcard
+
+| Field | Response |
+| --- | --- |
+| Model Application Fields | Industrial/Machinery and Robotics, Media & Entertainment. Generated motion is not validated for physical robot control. |
+| Life-critical impact | The model generates kinematic reference motion, not a safety-certified robot policy. Feeding unvalidated motion to physical hardware can cause collision, damage or injury. |
+| Use Case Restrictions | Use of this model is governed by the [NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/). See [License and Terms of Use](#license-and-terms-of-use) for separate software and sample-data terms. |
+| Model and dataset restrictions | The pretrained package does not provide the original training corpus or demo motion bank. Rights to model weights and separately supplied motion examples are distinct. No claim about original access-control practices is inferred from this package. |
+
+For physical execution, use a downstream controller with appropriate joint and
+actuator limits, collision and contact handling, fall protection and
+application-specific safety validation. The accompanying Newton integration
+is a kinematic preview and does not establish successful tracking on a
+physical G1.
+

g1/SHA256SUMS

@@ -0,0 +1,33 @@
+adcb83b61946ffe4ac974cc3d2e7c2aa8f9832dbe75e0c29e16d53b0ae0aa44c  README.md
+d8bd82393b308d27bfc3cb1670d4e4723794c3b349fa3b83435ce0f93e5b5065  manifest.json
+359a134cce0e863346024bc8fe37286666e2e35c98588267ef4bb989bd67280d  out/SHA256SUMS
+5f4bfa948eadb13a7b0d186ed2cba2e6f837e6709898b8e932aa345ff853515e  out/motionbricks_pose/version_1/checkpoints/model-step=2000000.json
+2154d80cac2beb4bb4074b4db92ae59d9fdfbc58651550c32941fe8527d28313  out/motionbricks_pose/version_1/checkpoints/model-step=2000000.safetensors
+273af770d328b458510ee6049fac8e38fa486c8792cc27b3515dddc094a7cd1f  out/motionbricks_pose/version_1/config.yaml
+3c482b9f810a9c4921d2e9fe46dae4e9e70d48ac9ea7d7b19a91fad2490d75e5  out/motionbricks_pose/version_1/hparams.yaml
+f70ba7887c5db6cd77739e573bbb734af0ec79a8cbabd9e435b7c72a5d040663  out/motionbricks_pose/version_1/meta.yaml
+6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0  out/motionbricks_pose/version_1/skeleton/joints.npy
+02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4  out/motionbricks_pose/version_1/skeleton/parents.npy
+5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050  out/motionbricks_pose/version_1/skeleton/skeleton.yaml
+ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea  out/motionbricks_pose/version_1/stats/motion/mean.npy
+fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7  out/motionbricks_pose/version_1/stats/motion/std.npy
+5df3e2521ba80997251c72bf2f5dd22cea301df458a407ab34d5e75f28855b5e  out/motionbricks_root/version_1/checkpoints/model-step=2000000.json
+da6ff2c05e45dfe4ac5e8e8362a6404f5c927fcddb18410e6804b61e55d4f2dc  out/motionbricks_root/version_1/checkpoints/model-step=2000000.safetensors
+b174f03a333f7f7857c3e2b8a32da517caddd198f228efeea2e203d046ce212a  out/motionbricks_root/version_1/config.yaml
+15dfadeab65b8ab1931b4e5257a2f9ebfa57e559510af1bd5baa96cfe2baeaf5  out/motionbricks_root/version_1/hparams.yaml
+c541e9d63b9b876a56c031a360d4d3929013b83515510a10f47ae63f6c7c8f17  out/motionbricks_root/version_1/meta.yaml
+6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0  out/motionbricks_root/version_1/skeleton/joints.npy
+02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4  out/motionbricks_root/version_1/skeleton/parents.npy
+5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050  out/motionbricks_root/version_1/skeleton/skeleton.yaml
+ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea  out/motionbricks_root/version_1/stats/motion/mean.npy
+fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7  out/motionbricks_root/version_1/stats/motion/std.npy
+ceaa8820137c55ddc472a499307e8fbf4f9e9e4c503e73b967fa43ae0afc2141  out/motionbricks_vqvae/version_1/checkpoints/model-step=2000000.json
+bf9c31173b334303bf392d184c24701ad1a095b3e94a5fe6b11db4d01343a454  out/motionbricks_vqvae/version_1/checkpoints/model-step=2000000.safetensors
+027a2d7ba5f49cadeadbcc9a6b0c6784d657d5a842e41ff820c5233f1cd6c1f3  out/motionbricks_vqvae/version_1/config.yaml
+7af860d6c144ff71ccdb4085b8413be892194ef419031b81e231502c2181576d  out/motionbricks_vqvae/version_1/hparams.yaml
+7bdf23605dcb89708a39bae33e12de59ebf4d216ba09b91931a2f813cc57e8c7  out/motionbricks_vqvae/version_1/meta.yaml
+6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0  out/motionbricks_vqvae/version_1/skeleton/joints.npy
+02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4  out/motionbricks_vqvae/version_1/skeleton/parents.npy
+5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050  out/motionbricks_vqvae/version_1/skeleton/skeleton.yaml
+ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea  out/motionbricks_vqvae/version_1/stats/motion/mean.npy
+fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7  out/motionbricks_vqvae/version_1/stats/motion/std.npy

g1/manifest.json

@@ -0,0 +1,181 @@
+{
+  "schema_version": 2,
+  "model_family": "MotionBricks",
+  "model_variant": "Unitree G1",
+  "checkpoint_step": 2000000,
+  "weight_format": "safetensors",
+  "weights_only": true,
+  "archive": {
+    "filename": "motionbricks_g1_checkpoints.tar.gz",
+    "bytes": 724645328,
+    "sha256": "66005b3f8a6e0aa57caaf40ef1701051af700e7b5b02eaacce073b7469c43cef",
+    "root_directory": "out"
+  },
+  "source_archive": {
+    "filename": "motionbricks_g1_checkpoints.tar.gz",
+    "bytes": 2134927408,
+    "sha256": "74e50e7f62367461f0c6c12fba9198863efc5135868fb306358ab74ad3b3bce9",
+    "root_directory": "out"
+  },
+  "original_checkpoint_bytes_preserved": false,
+  "original_model_tensor_bytes_preserved": true,
+  "skeleton_tensor_bytes_preserved": true,
+  "training_resume_state_included": false,
+  "files": [
+    {
+      "path": "out/motionbricks_pose/version_1/checkpoints/model-step=2000000.json",
+      "bytes": 346,
+      "sha256": "5f4bfa948eadb13a7b0d186ed2cba2e6f837e6709898b8e932aa345ff853515e"
+    },
+    {
+      "path": "out/motionbricks_pose/version_1/checkpoints/model-step=2000000.safetensors",
+      "bytes": 546380073,
+      "sha256": "2154d80cac2beb4bb4074b4db92ae59d9fdfbc58651550c32941fe8527d28313"
+    },
+    {
+      "path": "out/motionbricks_pose/version_1/config.yaml",
+      "bytes": 7243,
+      "sha256": "273af770d328b458510ee6049fac8e38fa486c8792cc27b3515dddc094a7cd1f"
+    },
+    {
+      "path": "out/motionbricks_pose/version_1/hparams.yaml",
+      "bytes": 7176,
+      "sha256": "3c482b9f810a9c4921d2e9fe46dae4e9e70d48ac9ea7d7b19a91fad2490d75e5"
+    },
+    {
+      "path": "out/motionbricks_pose/version_1/meta.yaml",
+      "bytes": 29,
+      "sha256": "f70ba7887c5db6cd77739e573bbb734af0ec79a8cbabd9e435b7c72a5d040663"
+    },
+    {
+      "path": "out/motionbricks_pose/version_1/skeleton/joints.npy",
+      "bytes": 944,
+      "sha256": "6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0"
+    },
+    {
+      "path": "out/motionbricks_pose/version_1/skeleton/parents.npy",
+      "bytes": 400,
+      "sha256": "02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4"
+    },
+    {
+      "path": "out/motionbricks_pose/version_1/skeleton/skeleton.yaml",
+      "bytes": 213,
+      "sha256": "5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050"
+    },
+    {
+      "path": "out/motionbricks_pose/version_1/stats/motion/mean.npy",
+      "bytes": 3472,
+      "sha256": "ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea"
+    },
+    {
+      "path": "out/motionbricks_pose/version_1/stats/motion/std.npy",
+      "bytes": 3472,
+      "sha256": "fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7"
+    },
+    {
+      "path": "out/motionbricks_root/version_1/checkpoints/model-step=2000000.json",
+      "bytes": 346,
+      "sha256": "5df3e2521ba80997251c72bf2f5dd22cea301df458a407ab34d5e75f28855b5e"
+    },
+    {
+      "path": "out/motionbricks_root/version_1/checkpoints/model-step=2000000.safetensors",
+      "bytes": 136510540,
+      "sha256": "da6ff2c05e45dfe4ac5e8e8362a6404f5c927fcddb18410e6804b61e55d4f2dc"
+    },
+    {
+      "path": "out/motionbricks_root/version_1/config.yaml",
+      "bytes": 6397,
+      "sha256": "b174f03a333f7f7857c3e2b8a32da517caddd198f228efeea2e203d046ce212a"
+    },
+    {
+      "path": "out/motionbricks_root/version_1/hparams.yaml",
+      "bytes": 6330,
+      "sha256": "15dfadeab65b8ab1931b4e5257a2f9ebfa57e559510af1bd5baa96cfe2baeaf5"
+    },
+    {
+      "path": "out/motionbricks_root/version_1/meta.yaml",
+      "bytes": 29,
+      "sha256": "c541e9d63b9b876a56c031a360d4d3929013b83515510a10f47ae63f6c7c8f17"
+    },
+    {
+      "path": "out/motionbricks_root/version_1/skeleton/joints.npy",
+      "bytes": 944,
+      "sha256": "6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0"
+    },
+    {
+      "path": "out/motionbricks_root/version_1/skeleton/parents.npy",
+      "bytes": 400,
+      "sha256": "02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4"
+    },
+    {
+      "path": "out/motionbricks_root/version_1/skeleton/skeleton.yaml",
+      "bytes": 213,
+      "sha256": "5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050"
+    },
+    {
+      "path": "out/motionbricks_root/version_1/stats/motion/mean.npy",
+      "bytes": 3472,
+      "sha256": "ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea"
+    },
+    {
+      "path": "out/motionbricks_root/version_1/stats/motion/std.npy",
+      "bytes": 3472,
+      "sha256": "fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7"
+    },
+    {
+      "path": "out/motionbricks_vqvae/version_1/checkpoints/model-step=2000000.json",
+      "bytes": 13245,
+      "sha256": "ceaa8820137c55ddc472a499307e8fbf4f9e9e4c503e73b967fa43ae0afc2141"
+    },
+    {
+      "path": "out/motionbricks_vqvae/version_1/checkpoints/model-step=2000000.safetensors",
+      "bytes": 95194080,
+      "sha256": "bf9c31173b334303bf392d184c24701ad1a095b3e94a5fe6b11db4d01343a454"
+    },
+    {
+      "path": "out/motionbricks_vqvae/version_1/config.yaml",
+      "bytes": 6320,
+      "sha256": "027a2d7ba5f49cadeadbcc9a6b0c6784d657d5a842e41ff820c5233f1cd6c1f3"
+    },
+    {
+      "path": "out/motionbricks_vqvae/version_1/hparams.yaml",
+      "bytes": 6251,
+      "sha256": "7af860d6c144ff71ccdb4085b8413be892194ef419031b81e231502c2181576d"
+    },
+    {
+      "path": "out/motionbricks_vqvae/version_1/meta.yaml",
+      "bytes": 30,
+      "sha256": "7bdf23605dcb89708a39bae33e12de59ebf4d216ba09b91931a2f813cc57e8c7"
+    },
+    {
+      "path": "out/motionbricks_vqvae/version_1/skeleton/joints.npy",
+      "bytes": 944,
+      "sha256": "6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0"
+    },
+    {
+      "path": "out/motionbricks_vqvae/version_1/skeleton/parents.npy",
+      "bytes": 400,
+      "sha256": "02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4"
+    },
+    {
+      "path": "out/motionbricks_vqvae/version_1/skeleton/skeleton.yaml",
+      "bytes": 213,
+      "sha256": "5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050"
+    },
+    {
+      "path": "out/motionbricks_vqvae/version_1/stats/motion/mean.npy",
+      "bytes": 3472,
+      "sha256": "ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea"
+    },
+    {
+      "path": "out/motionbricks_vqvae/version_1/stats/motion/std.npy",
+      "bytes": 3472,
+      "sha256": "fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7"
+    },
+    {
+      "path": "out/SHA256SUMS",
+      "bytes": 3493,
+      "sha256": "359a134cce0e863346024bc8fe37286666e2e35c98588267ef4bb989bd67280d"
+    }
+  ]
+}

g1/out/SHA256SUMS

@@ -0,0 +1,30 @@
+5f4bfa948eadb13a7b0d186ed2cba2e6f837e6709898b8e932aa345ff853515e  motionbricks_pose/version_1/checkpoints/model-step=2000000.json
+2154d80cac2beb4bb4074b4db92ae59d9fdfbc58651550c32941fe8527d28313  motionbricks_pose/version_1/checkpoints/model-step=2000000.safetensors
+273af770d328b458510ee6049fac8e38fa486c8792cc27b3515dddc094a7cd1f  motionbricks_pose/version_1/config.yaml
+3c482b9f810a9c4921d2e9fe46dae4e9e70d48ac9ea7d7b19a91fad2490d75e5  motionbricks_pose/version_1/hparams.yaml
+f70ba7887c5db6cd77739e573bbb734af0ec79a8cbabd9e435b7c72a5d040663  motionbricks_pose/version_1/meta.yaml
+6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0  motionbricks_pose/version_1/skeleton/joints.npy
+02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4  motionbricks_pose/version_1/skeleton/parents.npy
+5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050  motionbricks_pose/version_1/skeleton/skeleton.yaml
+ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea  motionbricks_pose/version_1/stats/motion/mean.npy
+fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7  motionbricks_pose/version_1/stats/motion/std.npy
+5df3e2521ba80997251c72bf2f5dd22cea301df458a407ab34d5e75f28855b5e  motionbricks_root/version_1/checkpoints/model-step=2000000.json
+da6ff2c05e45dfe4ac5e8e8362a6404f5c927fcddb18410e6804b61e55d4f2dc  motionbricks_root/version_1/checkpoints/model-step=2000000.safetensors
+b174f03a333f7f7857c3e2b8a32da517caddd198f228efeea2e203d046ce212a  motionbricks_root/version_1/config.yaml
+15dfadeab65b8ab1931b4e5257a2f9ebfa57e559510af1bd5baa96cfe2baeaf5  motionbricks_root/version_1/hparams.yaml
+c541e9d63b9b876a56c031a360d4d3929013b83515510a10f47ae63f6c7c8f17  motionbricks_root/version_1/meta.yaml
+6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0  motionbricks_root/version_1/skeleton/joints.npy
+02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4  motionbricks_root/version_1/skeleton/parents.npy
+5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050  motionbricks_root/version_1/skeleton/skeleton.yaml
+ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea  motionbricks_root/version_1/stats/motion/mean.npy
+fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7  motionbricks_root/version_1/stats/motion/std.npy
+ceaa8820137c55ddc472a499307e8fbf4f9e9e4c503e73b967fa43ae0afc2141  motionbricks_vqvae/version_1/checkpoints/model-step=2000000.json
+bf9c31173b334303bf392d184c24701ad1a095b3e94a5fe6b11db4d01343a454  motionbricks_vqvae/version_1/checkpoints/model-step=2000000.safetensors
+027a2d7ba5f49cadeadbcc9a6b0c6784d657d5a842e41ff820c5233f1cd6c1f3  motionbricks_vqvae/version_1/config.yaml
+7af860d6c144ff71ccdb4085b8413be892194ef419031b81e231502c2181576d  motionbricks_vqvae/version_1/hparams.yaml
+7bdf23605dcb89708a39bae33e12de59ebf4d216ba09b91931a2f813cc57e8c7  motionbricks_vqvae/version_1/meta.yaml
+6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0  motionbricks_vqvae/version_1/skeleton/joints.npy
+02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4  motionbricks_vqvae/version_1/skeleton/parents.npy
+5820a223977a36e04e75824a7f720b57225ac30a067bdd32bc312469f3644050  motionbricks_vqvae/version_1/skeleton/skeleton.yaml
+ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea  motionbricks_vqvae/version_1/stats/motion/mean.npy
+fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7  motionbricks_vqvae/version_1/stats/motion/std.npy

g1/out/motionbricks_pose/version_1/checkpoints/model-step=2000000.json

@@ -0,0 +1,14 @@
+{
+  "schema_version": 1,
+  "source_checkpoint_sha256": "0223c352b308ba638a499cc5c92104da36cb1d04cea2f8ce61d54a2489f853f1",
+  "epoch": 25973,
+  "global_step": 2000000,
+  "pytorch_lightning_version": "2.4.0",
+  "state_dict_metadata": {},
+  "omitted_training_state": [
+    "loops",
+    "callbacks",
+    "optimizer_states",
+    "lr_schedulers"
+  ]
+}

g1/out/motionbricks_pose/version_1/checkpoints/model-step=2000000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2154d80cac2beb4bb4074b4db92ae59d9fdfbc58651550c32941fe8527d28313
+size 546380073

g1/out/motionbricks_pose/version_1/config.yaml

@@ -0,0 +1,245 @@
+model:
+  backbone_network:
+    _target_: motionbricks.motion_backbone.neural_modules.pose_backbone.pose_backbone_network
+    motion_rep: ???
+    args:
+      pose_root_mode: ${model.args.pose_root_mode}
+      min_tokens: ${model.args.min_tokens}
+      max_tokens: ${model.args.max_tokens}
+      down_t: ${model.args.down_t}
+      pose_token_mlp_num_layers: 2
+      root_feat_width: 256
+      pose_feat_width: 640
+      token_length_feat_width: 128
+      text_emb_dim: 4096
+      text_embeddings: ${data.text_embeddings}
+      local_pose_feature: ${model.args.local_pose_feature}
+      cond_root_feature: ${model.args.cond_root_feature}
+      cond_root_feature_is_from_motion_rep: ${model.args.cond_root_feature_is_from_motion_rep}
+      n_embd: 1024
+      n_head: 16
+      n_layers: 16
+      pose_vqvae:
+        nb_code: ${model.pose_vqvae_network.nb_code}
+        code_dim: ${model.pose_vqvae_network.code_dim}
+        num_heads: ${model.pose_vqvae_network.num_heads}
+        has_codebook: ${oc.select:model.pose_vqvae_network.has_codebook,true}
+      root_vqvae:
+        nb_code: 1024
+        code_dim: 60
+        num_heads: 1
+  pose_vqvae_network:
+    _target_: motionbricks.vqvae.neural_modules.vqvae.VQVAE
+    pose_root_mode: pose
+    motion_rep: ???
+    encoder_state_dim: 241
+    decoder_state_dim: 329
+    decoder_target_cond_dim: 241
+    decoder_external_cond_dim: 2
+    feature_mode:
+    - joint_positions_and_rotations_and_hip_height
+    - pose
+    - joint_positions_and_rotations_and_hip_height
+    - root_without_hip_height_without_heading
+    quantizer_strategy: multihead_ema_reset
+    quantizer_mu: 0.99
+    nb_code: 100000000
+    code_dim: 256
+    output_emb_width: 256
+    down_t: 2
+    stride_t: 2
+    width: 512
+    depth: 4
+    dilation_growth_rate: 3
+    activation: relu
+    num_heads: 8
+    kmeans_init: false
+    norm: None
+    calculate_per_head_perplexity: true
+    cond_fusion_last_layer: false
+  _target_: motionbricks.motion_backbone.models.pose_model.MotionModel
+  motion_rep: ???
+  root_vqvae_network: null
+  pose_vqvae_motion_rep: local
+  denoiser:
+    backbone:
+      llm_shape: null
+  args:
+    min_tokens: 6
+    max_tokens: 16
+    down_t: 2
+    vqvae_model_ckpt_path: out/motionbricks_vqvae/version_1/checkpoints/model-step=2000000.ckpt
+    pose_vqvae_motion_rep: ${model.pose_vqvae_motion_rep}
+    incorrect_token_ratio_min: 0.0
+    incorrect_token_ratio_max: 1.0
+    masked_token_ratio: 0.8
+    batchsize_mul_factor: 2
+    pose_root_mode: pose
+    local_pose_feature: joint_positions_and_rotations_and_hip_height
+    cond_root_feature: root_without_hip_height
+    cond_root_feature_is_from_motion_rep: global
+    max_num_start_keyframes: 4
+    no_start_keyframe_prob: 0.0
+    max_num_end_keyframes: 4
+    no_end_keyframe_prob: 0.0
+    max_num_middle_keyframes: 8
+    no_middle_keyframe_prob: 0.5
+    prob_provide_text_emb: 0.2
+    keyframe_num_warmup_steps: 200000
+    floor_estimation: min_joint_height_within_windows
+    floor_estimation_window_time: 1.0
+  optimizer:
+    _target_: adam_atan2_pytorch.AdamAtan2
+    _partial_: true
+    lr: 0.0001
+    weight_decay: 0.0
+  scheduler:
+    _target_: motionbricks.motionlib.train.scheduler.WarmupCosineScheduler
+    _partial_: true
+    num_warmup_steps: 10000
+    num_training_steps: ${trainer.max_steps}
+    final_lr: 2.0e-06
+    last_epoch: -1
+    lt_kwargs:
+      interval: step
+      frequency: 1
+trainer:
+  _target_: pytorch_lightning.Trainer
+  precision: 32
+  enable_progress_bar: false
+  profiler: null
+  check_val_every_n_epoch: null
+  detect_anomaly: false
+  num_sanity_val_steps: 0
+  max_steps: 2000001
+  gradient_clip_val: 0.5
+  log_every_n_steps: 50
+  val_check_interval: 50000
+  devices: 8
+  num_nodes: 4
+  accelerator: gpu
+  strategy: ddp
+loggers:
+  out_dir: ${out_dir}
+  cfg_id: ${id}
+  max_steps: ${trainer.max_steps}
+  log_nsteps: ${trainer.log_every_n_steps}
+  use_wandb: true
+  wandb_run: ${wandb_run}
+  wandb_group: null
+  wandb_project: motionbricks
+  wandb_entity: null
+  wandb_run_name: motionbricks_pose
+  logger_project: motionbricks
+callbacks:
+  callback_dict:
+    checkpoint_epoch_cb:
+      _target_: motionbricks.motionlib.train.callbacks.ckpt.ReadPermissionModelCheckpoint
+      monitor: null
+      dirpath: out/motionbricks_pose/version_1/checkpoints
+      filename: model-{step:07d}
+      save_last: false
+      save_top_k: -1
+      mode: min
+      every_n_train_steps: 50000
+    learning_rate_monitor:
+      _target_: pytorch_lightning.callbacks.LearningRateMonitor
+      logging_interval: step
+    autoresume:
+      _target_: motionbricks.motionlib.train.callbacks.autoresume.AutoResumeCallback
+      autoresume_after: 1260000
+      save_checkpoint_on_exception: true
+    pose_evaluate:
+      _target_: motionbricks.motion_backbone.callbacks.evaluation.PoseEvaluate
+      every_n_steps: 50001
+      train_dataloader: ???
+      val_dataloader: ???
+      seed: 2333
+      batch_size: 32
+      max_num_data: 40000
+      max_num_visualization: 32
+      out_dir: ${run_dir}/vis
+  only_rank_zero:
+  - learning_rate_monitor
+  - pose_evaluate
+  need_version: []
+  need_dataloaders:
+  - pose_evaluate
+  need_ema: []
+  need_motionbricks_test: []
+  need_log:
+  - pose_evaluate
+  need_autoresume:
+  - pose_evaluate
+  need_mask_cond_sampler: []
+  need_one_logger_callback:
+  - autoresume
+dataloader:
+  _target_: torch.utils.data.DataLoader
+  batch_size: 128
+  num_workers: 11
+  shuffle: ???
+  persistent_workers: true
+  collate_fn:
+    _target_: motionbricks.motionlib.data.utils.collate_batch
+    _partial_: true
+    llm_shape: ${model.denoiser.backbone.llm_shape}
+    num_frames: null
+data:
+  _target_: motionbricks.motionlib.data.motion_dataset.MotionDataset
+  name: motionbricks-G1
+  folder: ../datasets/motionbricks-G1
+  text_embeddings: null
+  split: ???
+  motion_sampler:
+    _target_: motionbricks.motionlib.data.motion_sampler.MaxDurationRandomCrop
+    max_seconds: 30
+  motion_loading_mode: memmap
+  use_natural_desc: true
+  use_short_desc: true
+  use_technical_desc: false
+  use_nv_overview_desc: false
+  nv_overview_prob: null
+  drop_text_prob: 0.1
+  augment_text: false
+  aug_text_suffix: ''
+  aug_text_prob: 0.75
+  aug_text_ind_range:
+  - 1
+  - 61
+  load_neutral_joints: false
+  timelines_mode: false
+  timelines_name: null
+  dataset_sampler: null
+  dataset_sampler_only_at_train: true
+  to_canonicalize: false
+  randomize_first_heading: true
+  loading_mode: motion_only
+skeleton:
+  base_name: g1skel34
+  folder: out/motionbricks_pose/version_1/skeleton
+  t_pose: capture
+  orig_prefix: ''
+  name: g1skel34
+  _target_: motionbricks.motionlib.core.skeletons.g1.G1Skeleton34
+motion_rep:
+  name: ${skeleton.name}_dual_root_global_joints
+  stats:
+    _target_: motionbricks.motionlib.core.utils.stats.Stats
+    folder: out/motionbricks_pose/version_1/stats/motion
+  _target_: motionbricks.motionlib.core.motion_reps.dual_root_global_joints.GlobalRootGlobalJoints
+fps: 30
+wandb_run: motionbricks_pose
+resume: false
+version: 1
+cp: last_epoch
+matmul_precision: high
+seed: 7
+id: ${hydra:runtime.choices.exp}_gpu${trainer.devices}_node${trainer.num_nodes}
+out_dir: out/motionbricks_pose
+run_dir: out/motionbricks_pose/version_1
+pure_testing: false
+config_version: 4
+data_root: datasets
+tmr_checkpoints_root: null
+use_one_logger: false

g1/out/motionbricks_pose/version_1/hparams.yaml

@@ -0,0 +1,245 @@
+model:
+  backbone_network:
+    _target_: motionbricks.motion_backbone.neural_modules.pose_backbone.pose_backbone_network
+    motion_rep: ???
+    args:
+      pose_root_mode: ${model.args.pose_root_mode}
+      min_tokens: ${model.args.min_tokens}
+      max_tokens: ${model.args.max_tokens}
+      down_t: ${model.args.down_t}
+      pose_token_mlp_num_layers: 2
+      root_feat_width: 256
+      pose_feat_width: 640
+      token_length_feat_width: 128
+      text_emb_dim: 4096
+      text_embeddings: ${data.text_embeddings}
+      local_pose_feature: ${model.args.local_pose_feature}
+      cond_root_feature: ${model.args.cond_root_feature}
+      cond_root_feature_is_from_motion_rep: ${model.args.cond_root_feature_is_from_motion_rep}
+      n_embd: 1024
+      n_head: 16
+      n_layers: 16
+      pose_vqvae:
+        nb_code: ${model.pose_vqvae_network.nb_code}
+        code_dim: ${model.pose_vqvae_network.code_dim}
+        num_heads: ${model.pose_vqvae_network.num_heads}
+        has_codebook: ${oc.select:model.pose_vqvae_network.has_codebook,true}
+      root_vqvae:
+        nb_code: 1024
+        code_dim: 60
+        num_heads: 1
+  pose_vqvae_network:
+    _target_: motionbricks.vqvae.neural_modules.vqvae.VQVAE
+    pose_root_mode: pose
+    motion_rep: ???
+    encoder_state_dim: 241
+    decoder_state_dim: 329
+    decoder_target_cond_dim: 241
+    decoder_external_cond_dim: 2
+    feature_mode:
+    - joint_positions_and_rotations_and_hip_height
+    - pose
+    - joint_positions_and_rotations_and_hip_height
+    - root_without_hip_height_without_heading
+    quantizer_strategy: multihead_ema_reset
+    quantizer_mu: 0.99
+    nb_code: 100000000
+    code_dim: 256
+    output_emb_width: 256
+    down_t: 2
+    stride_t: 2
+    width: 512
+    depth: 4
+    dilation_growth_rate: 3
+    activation: relu
+    num_heads: 8
+    kmeans_init: false
+    norm: None
+    calculate_per_head_perplexity: true
+    cond_fusion_last_layer: false
+  _target_: motionbricks.motion_backbone.models.pose_model.MotionModel
+  motion_rep: ???
+  root_vqvae_network: null
+  pose_vqvae_motion_rep: local
+  denoiser:
+    backbone:
+      llm_shape: null
+  args:
+    min_tokens: 6
+    max_tokens: 16
+    down_t: 2
+    vqvae_model_ckpt_path: out/motionbricks_vqvae/version_1/checkpoints/model-step=2000000.ckpt
+    pose_vqvae_motion_rep: ${model.pose_vqvae_motion_rep}
+    incorrect_token_ratio_min: 0.0
+    incorrect_token_ratio_max: 1.0
+    masked_token_ratio: 0.8
+    batchsize_mul_factor: 2
+    pose_root_mode: pose
+    local_pose_feature: joint_positions_and_rotations_and_hip_height
+    cond_root_feature: root_without_hip_height
+    cond_root_feature_is_from_motion_rep: global
+    max_num_start_keyframes: 4
+    no_start_keyframe_prob: 0.0
+    max_num_end_keyframes: 4
+    no_end_keyframe_prob: 0.0
+    max_num_middle_keyframes: 8
+    no_middle_keyframe_prob: 0.5
+    prob_provide_text_emb: 0.2
+    keyframe_num_warmup_steps: 200000
+    floor_estimation: min_joint_height_within_windows
+    floor_estimation_window_time: 1.0
+  optimizer:
+    _target_: adam_atan2_pytorch.AdamAtan2
+    _partial_: true
+    lr: 0.0001
+    weight_decay: 0.0
+  scheduler:
+    _target_: motionbricks.motionlib.train.scheduler.WarmupCosineScheduler
+    _partial_: true
+    num_warmup_steps: 10000
+    num_training_steps: ${trainer.max_steps}
+    final_lr: 2.0e-06
+    last_epoch: -1
+    lt_kwargs:
+      interval: step
+      frequency: 1
+trainer:
+  _target_: pytorch_lightning.Trainer
+  precision: 32
+  enable_progress_bar: false
+  profiler: null
+  check_val_every_n_epoch: null
+  detect_anomaly: false
+  num_sanity_val_steps: 0
+  max_steps: 2000001
+  gradient_clip_val: 0.5
+  log_every_n_steps: 50
+  val_check_interval: 50000
+  devices: 8
+  num_nodes: 4
+  accelerator: gpu
+  strategy: ddp
+loggers:
+  out_dir: ${out_dir}
+  cfg_id: ${id}
+  max_steps: ${trainer.max_steps}
+  log_nsteps: ${trainer.log_every_n_steps}
+  use_wandb: true
+  wandb_run: ${wandb_run}
+  wandb_group: null
+  wandb_project: motionbricks
+  wandb_entity: null
+  wandb_run_name: motionbricks_pose
+  logger_project: motionbricks
+callbacks:
+  callback_dict:
+    checkpoint_epoch_cb:
+      _target_: motionbricks.motionlib.train.callbacks.ckpt.ReadPermissionModelCheckpoint
+      monitor: null
+      dirpath: null
+      filename: model-{step:07d}
+      save_last: false
+      save_top_k: -1
+      mode: min
+      every_n_train_steps: 50000
+    learning_rate_monitor:
+      _target_: pytorch_lightning.callbacks.LearningRateMonitor
+      logging_interval: step
+    autoresume:
+      _target_: motionbricks.motionlib.train.callbacks.autoresume.AutoResumeCallback
+      autoresume_after: 1260000
+      save_checkpoint_on_exception: true
+    pose_evaluate:
+      _target_: motionbricks.motion_backbone.callbacks.evaluation.PoseEvaluate
+      every_n_steps: 50001
+      train_dataloader: ???
+      val_dataloader: ???
+      seed: 2333
+      batch_size: 32
+      max_num_data: 40000
+      max_num_visualization: 32
+      out_dir: ${run_dir}/vis
+  only_rank_zero:
+  - learning_rate_monitor
+  - pose_evaluate
+  need_version: []
+  need_dataloaders:
+  - pose_evaluate
+  need_ema: []
+  need_motionbricks_test: []
+  need_log:
+  - pose_evaluate
+  need_autoresume:
+  - pose_evaluate
+  need_mask_cond_sampler: []
+  need_one_logger_callback:
+  - autoresume
+dataloader:
+  _target_: torch.utils.data.DataLoader
+  batch_size: 128
+  num_workers: 11
+  shuffle: ???
+  persistent_workers: true
+  collate_fn:
+    _target_: motionbricks.motionlib.data.utils.collate_batch
+    _partial_: true
+    llm_shape: ${model.denoiser.backbone.llm_shape}
+    num_frames: null
+data:
+  _target_: motionbricks.motionlib.data.motion_dataset.MotionDataset
+  name: motionbricks-G1
+  folder: ../datasets/motionbricks-G1
+  text_embeddings: null
+  split: ???
+  motion_sampler:
+    _target_: motionbricks.motionlib.data.motion_sampler.MaxDurationRandomCrop
+    max_seconds: 30
+  motion_loading_mode: memmap
+  use_natural_desc: true
+  use_short_desc: true
+  use_technical_desc: false
+  use_nv_overview_desc: false
+  nv_overview_prob: null
+  drop_text_prob: 0.1
+  augment_text: false
+  aug_text_suffix: ''
+  aug_text_prob: 0.75
+  aug_text_ind_range:
+  - 1
+  - 61
+  load_neutral_joints: false
+  timelines_mode: false
+  timelines_name: null
+  dataset_sampler: null
+  dataset_sampler_only_at_train: true
+  to_canonicalize: false
+  randomize_first_heading: true
+  loading_mode: motion_only
+skeleton:
+  base_name: g1skel34
+  folder: out/motionbricks_pose/version_1/skeleton
+  t_pose: capture
+  orig_prefix: ''
+  name: g1skel34
+  _target_: motionbricks.motionlib.core.skeletons.g1.G1Skeleton34
+motion_rep:
+  name: ${skeleton.name}_dual_root_global_joints
+  stats:
+    _target_: motionbricks.motionlib.core.utils.stats.Stats
+    folder: out/motionbricks_pose/version_1/stats/motion
+  _target_: motionbricks.motionlib.core.motion_reps.dual_root_global_joints.GlobalRootGlobalJoints
+fps: 30
+wandb_run: motionbricks_pose
+resume: false
+version: 1
+cp: last_epoch
+matmul_precision: high
+seed: 7
+id: ${hydra:runtime.choices.exp}_gpu${trainer.devices}_node${trainer.num_nodes}
+out_dir: out/motionbricks_pose
+run_dir: ???
+pure_testing: false
+config_version: 4
+data_root: datasets
+tmr_checkpoints_root: null
+use_one_logger: false

g1/out/motionbricks_pose/version_1/meta.yaml

@@ -0,0 +1 @@
+wandb_run: motionbricks_pose

g1/out/motionbricks_pose/version_1/skeleton/joints.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0
+size 944

g1/out/motionbricks_pose/version_1/skeleton/parents.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4
+size 400

g1/out/motionbricks_pose/version_1/skeleton/skeleton.yaml

@@ -0,0 +1,6 @@
+base_name: g1skel34
+folder: ${data.folder}/features/motion_rep/${motion_rep.name}_fps_${fps}/skeleton
+t_pose: capture
+orig_prefix: ''
+name: g1skel34
+_target_: motionbricks.motionlib.core.skeletons.g1.G1Skeleton34

g1/out/motionbricks_pose/version_1/stats/motion/mean.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea
+size 3472

g1/out/motionbricks_pose/version_1/stats/motion/std.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7
+size 3472

g1/out/motionbricks_root/version_1/checkpoints/model-step=2000000.json

@@ -0,0 +1,14 @@
+{
+  "schema_version": 1,
+  "source_checkpoint_sha256": "d7299a9b1f5aca35730c36dfe7ea28075708ac8266bf384fe8e2ef9c9aee69c7",
+  "epoch": 13071,
+  "global_step": 2000000,
+  "pytorch_lightning_version": "2.4.0",
+  "state_dict_metadata": {},
+  "omitted_training_state": [
+    "loops",
+    "callbacks",
+    "optimizer_states",
+    "lr_schedulers"
+  ]
+}

g1/out/motionbricks_root/version_1/checkpoints/model-step=2000000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da6ff2c05e45dfe4ac5e8e8362a6404f5c927fcddb18410e6804b61e55d4f2dc
+size 136510540

g1/out/motionbricks_root/version_1/config.yaml

@@ -0,0 +1,223 @@
+model:
+  backbone_network:
+    _target_: motionbricks.motion_backbone.neural_modules.root_backbone.root_backbone_network
+    motion_rep: ???
+    args:
+      min_tokens: ${model.args.min_tokens}
+      max_tokens: ${model.args.max_tokens}
+      down_t: ${model.args.down_t}
+      use_hard_num_token_emb_for_root_prediction: ${model.args.use_hard_num_token_emb_for_root_prediction}
+      local_root_feature: ${model.args.local_root_feature}
+      global_root_feature: ${model.args.global_root_feature}
+      local_pose_feature: ${model.args.local_pose_feature}
+      pose_feat_dim: 256
+      local_root_feat_dim: 64
+      global_root_feat_dim: 64
+      text_emb_dim: 4096
+      text_embeddings: ${data.text_embeddings}
+      input_feat_mlp_num_layers: 2
+      n_embd: 512
+      n_head: 16
+      n_layers_shared: 3
+      n_layers_root_token: 3
+      width: 512
+      depth: 4
+      dilation_growth_rate: 3
+      activation: relu
+      norm: None
+      pose_vqvae:
+        nb_code: 1024
+        code_dim: 60
+        num_heads: 1
+      root_vqvae:
+        nb_code: 1024
+        code_dim: 60
+        num_heads: 1
+  _target_: motionbricks.motion_backbone.models.root_model.MotionModel
+  motion_rep: ???
+  root_vqvae_network: null
+  pose_vqvae_network: null
+  denoiser:
+    backbone:
+      llm_shape: null
+  args:
+    pose_root_mode: root
+    local_pose_feature: joint_positions_and_rotations_and_hip_height
+    local_root_feature: root
+    global_root_feature: root
+    min_tokens: 6
+    max_tokens: 16
+    use_hard_num_token_emb_for_root_prediction: true
+    min_off_target_tokens_to_sample: 25
+    max_off_target_tokens_to_sample: 40
+    prob_off_range_target_token: 0.0
+    prob_provide_num_tokens: 0.0
+    down_t: 2
+    vqvae_model_ckpt_path: null
+    batchsize_mul_factor: 2
+    max_num_start_keyframes: 4
+    no_start_keyframe_prob: 0.0
+    max_num_end_keyframes: 4
+    no_end_keyframe_prob: 0.0
+    prob_provide_text_emb: 0.0
+    num_token_loss_coeff: 1.0
+    global_root_loss_coeff: 2.0
+    local_root_loss_coeff: 1.0
+    keyframe_num_warmup_steps: 200000
+    floor_estimation: min_joint_height_within_windows
+    floor_estimation_window_time: 1.0
+  optimizer:
+    _target_: adam_atan2_pytorch.AdamAtan2
+    _partial_: true
+    lr: 0.0001
+    weight_decay: 0.0
+  scheduler:
+    _target_: motionbricks.motionlib.train.scheduler.WarmupCosineScheduler
+    _partial_: true
+    num_warmup_steps: 10000
+    num_training_steps: ${trainer.max_steps}
+    final_lr: 2.0e-06
+    last_epoch: -1
+    lt_kwargs:
+      interval: step
+      frequency: 1
+trainer:
+  _target_: pytorch_lightning.Trainer
+  precision: 32
+  enable_progress_bar: false
+  profiler: null
+  check_val_every_n_epoch: null
+  detect_anomaly: false
+  num_sanity_val_steps: 0
+  max_steps: 2000001
+  gradient_clip_val: 0.5
+  log_every_n_steps: 50
+  val_check_interval: 50000
+  devices: 8
+  num_nodes: 2
+  accelerator: gpu
+  strategy: ddp
+loggers:
+  out_dir: ${out_dir}
+  cfg_id: ${id}
+  max_steps: ${trainer.max_steps}
+  log_nsteps: ${trainer.log_every_n_steps}
+  use_wandb: true
+  wandb_run: ${wandb_run}
+  wandb_group: null
+  wandb_project: motionbricks
+  wandb_entity: null
+  wandb_run_name: motionbricks_root
+  logger_project: motionbricks
+callbacks:
+  callback_dict:
+    checkpoint_epoch_cb:
+      _target_: motionbricks.motionlib.train.callbacks.ckpt.ReadPermissionModelCheckpoint
+      monitor: null
+      dirpath: out/motionbricks_root/version_1/checkpoints
+      filename: model-{step:07d}
+      save_last: false
+      save_top_k: -1
+      mode: min
+      every_n_train_steps: 50000
+    learning_rate_monitor:
+      _target_: pytorch_lightning.callbacks.LearningRateMonitor
+      logging_interval: step
+    autoresume:
+      _target_: motionbricks.motionlib.train.callbacks.autoresume.AutoResumeCallback
+      autoresume_after: 1260000
+      save_checkpoint_on_exception: true
+    pose_evaluate:
+      _target_: motionbricks.motion_backbone.callbacks.evaluation.PoseEvaluate
+      every_n_steps: 50001
+      train_dataloader: ???
+      val_dataloader: ???
+      seed: 2333
+      batch_size: 32
+      max_num_data: 40000
+      max_num_visualization: 32
+      out_dir: ${run_dir}/vis
+  only_rank_zero:
+  - learning_rate_monitor
+  - pose_evaluate
+  need_version: []
+  need_dataloaders:
+  - pose_evaluate
+  need_ema: []
+  need_motionbricks_test: []
+  need_log:
+  - pose_evaluate
+  need_autoresume:
+  - pose_evaluate
+  need_mask_cond_sampler: []
+  need_one_logger_callback:
+  - autoresume
+dataloader:
+  _target_: torch.utils.data.DataLoader
+  batch_size: 128
+  num_workers: 11
+  shuffle: ???
+  persistent_workers: true
+  collate_fn:
+    _target_: motionbricks.motionlib.data.utils.collate_batch
+    _partial_: true
+    llm_shape: ${model.denoiser.backbone.llm_shape}
+    num_frames: null
+data:
+  _target_: motionbricks.motionlib.data.motion_dataset.MotionDataset
+  name: motionbricks-G1
+  folder: ../datasets/motionbricks-G1
+  text_embeddings: null
+  split: ???
+  motion_sampler:
+    _target_: motionbricks.motionlib.data.motion_sampler.MaxDurationRandomCrop
+    max_seconds: 30
+  motion_loading_mode: memmap
+  use_natural_desc: true
+  use_short_desc: true
+  use_technical_desc: false
+  use_nv_overview_desc: false
+  nv_overview_prob: null
+  drop_text_prob: 0.1
+  augment_text: false
+  aug_text_suffix: ''
+  aug_text_prob: 0.75
+  aug_text_ind_range:
+  - 1
+  - 61
+  load_neutral_joints: false
+  timelines_mode: false
+  timelines_name: null
+  dataset_sampler: null
+  dataset_sampler_only_at_train: true
+  to_canonicalize: false
+  randomize_first_heading: true
+  loading_mode: motion_only
+skeleton:
+  base_name: g1skel34
+  folder: out/motionbricks_root/version_1/skeleton
+  t_pose: capture
+  orig_prefix: ''
+  name: g1skel34
+  _target_: motionbricks.motionlib.core.skeletons.g1.G1Skeleton34
+motion_rep:
+  name: ${skeleton.name}_dual_root_global_joints
+  stats:
+    _target_: motionbricks.motionlib.core.utils.stats.Stats
+    folder: out/motionbricks_root/version_1/stats/motion
+  _target_: motionbricks.motionlib.core.motion_reps.dual_root_global_joints.GlobalRootGlobalJoints
+fps: 30
+wandb_run: motionbricks_root
+resume: false
+version: 1
+cp: last_epoch
+matmul_precision: high
+seed: 7
+id: ${hydra:runtime.choices.exp}_gpu${trainer.devices}_node${trainer.num_nodes}
+out_dir: out/motionbricks_root
+run_dir: out/motionbricks_root/version_1
+pure_testing: false
+config_version: 4
+data_root: datasets
+tmr_checkpoints_root: null
+use_one_logger: false

g1/out/motionbricks_root/version_1/hparams.yaml

@@ -0,0 +1,223 @@
+model:
+  backbone_network:
+    _target_: motionbricks.motion_backbone.neural_modules.root_backbone.root_backbone_network
+    motion_rep: ???
+    args:
+      min_tokens: ${model.args.min_tokens}
+      max_tokens: ${model.args.max_tokens}
+      down_t: ${model.args.down_t}
+      use_hard_num_token_emb_for_root_prediction: ${model.args.use_hard_num_token_emb_for_root_prediction}
+      local_root_feature: ${model.args.local_root_feature}
+      global_root_feature: ${model.args.global_root_feature}
+      local_pose_feature: ${model.args.local_pose_feature}
+      pose_feat_dim: 256
+      local_root_feat_dim: 64
+      global_root_feat_dim: 64
+      text_emb_dim: 4096
+      text_embeddings: ${data.text_embeddings}
+      input_feat_mlp_num_layers: 2
+      n_embd: 512
+      n_head: 16
+      n_layers_shared: 3
+      n_layers_root_token: 3
+      width: 512
+      depth: 4
+      dilation_growth_rate: 3
+      activation: relu
+      norm: None
+      pose_vqvae:
+        nb_code: 1024
+        code_dim: 60
+        num_heads: 1
+      root_vqvae:
+        nb_code: 1024
+        code_dim: 60
+        num_heads: 1
+  _target_: motionbricks.motion_backbone.models.root_model.MotionModel
+  motion_rep: ???
+  root_vqvae_network: null
+  pose_vqvae_network: null
+  denoiser:
+    backbone:
+      llm_shape: null
+  args:
+    pose_root_mode: root
+    local_pose_feature: joint_positions_and_rotations_and_hip_height
+    local_root_feature: root
+    global_root_feature: root
+    min_tokens: 6
+    max_tokens: 16
+    use_hard_num_token_emb_for_root_prediction: true
+    min_off_target_tokens_to_sample: 25
+    max_off_target_tokens_to_sample: 40
+    prob_off_range_target_token: 0.0
+    prob_provide_num_tokens: 0.0
+    down_t: 2
+    vqvae_model_ckpt_path: null
+    batchsize_mul_factor: 2
+    max_num_start_keyframes: 4
+    no_start_keyframe_prob: 0.0
+    max_num_end_keyframes: 4
+    no_end_keyframe_prob: 0.0
+    prob_provide_text_emb: 0.0
+    num_token_loss_coeff: 1.0
+    global_root_loss_coeff: 2.0
+    local_root_loss_coeff: 1.0
+    keyframe_num_warmup_steps: 200000
+    floor_estimation: min_joint_height_within_windows
+    floor_estimation_window_time: 1.0
+  optimizer:
+    _target_: adam_atan2_pytorch.AdamAtan2
+    _partial_: true
+    lr: 0.0001
+    weight_decay: 0.0
+  scheduler:
+    _target_: motionbricks.motionlib.train.scheduler.WarmupCosineScheduler
+    _partial_: true
+    num_warmup_steps: 10000
+    num_training_steps: ${trainer.max_steps}
+    final_lr: 2.0e-06
+    last_epoch: -1
+    lt_kwargs:
+      interval: step
+      frequency: 1
+trainer:
+  _target_: pytorch_lightning.Trainer
+  precision: 32
+  enable_progress_bar: false
+  profiler: null
+  check_val_every_n_epoch: null
+  detect_anomaly: false
+  num_sanity_val_steps: 0
+  max_steps: 2000001
+  gradient_clip_val: 0.5
+  log_every_n_steps: 50
+  val_check_interval: 50000
+  devices: 8
+  num_nodes: 2
+  accelerator: gpu
+  strategy: ddp
+loggers:
+  out_dir: ${out_dir}
+  cfg_id: ${id}
+  max_steps: ${trainer.max_steps}
+  log_nsteps: ${trainer.log_every_n_steps}
+  use_wandb: true
+  wandb_run: ${wandb_run}
+  wandb_group: null
+  wandb_project: motionbricks
+  wandb_entity: null
+  wandb_run_name: motionbricks_root
+  logger_project: motionbricks
+callbacks:
+  callback_dict:
+    checkpoint_epoch_cb:
+      _target_: motionbricks.motionlib.train.callbacks.ckpt.ReadPermissionModelCheckpoint
+      monitor: null
+      dirpath: null
+      filename: model-{step:07d}
+      save_last: false
+      save_top_k: -1
+      mode: min
+      every_n_train_steps: 50000
+    learning_rate_monitor:
+      _target_: pytorch_lightning.callbacks.LearningRateMonitor
+      logging_interval: step
+    autoresume:
+      _target_: motionbricks.motionlib.train.callbacks.autoresume.AutoResumeCallback
+      autoresume_after: 1260000
+      save_checkpoint_on_exception: true
+    pose_evaluate:
+      _target_: motionbricks.motion_backbone.callbacks.evaluation.PoseEvaluate
+      every_n_steps: 50001
+      train_dataloader: ???
+      val_dataloader: ???
+      seed: 2333
+      batch_size: 32
+      max_num_data: 40000
+      max_num_visualization: 32
+      out_dir: ${run_dir}/vis
+  only_rank_zero:
+  - learning_rate_monitor
+  - pose_evaluate
+  need_version: []
+  need_dataloaders:
+  - pose_evaluate
+  need_ema: []
+  need_motionbricks_test: []
+  need_log:
+  - pose_evaluate
+  need_autoresume:
+  - pose_evaluate
+  need_mask_cond_sampler: []
+  need_one_logger_callback:
+  - autoresume
+dataloader:
+  _target_: torch.utils.data.DataLoader
+  batch_size: 128
+  num_workers: 11
+  shuffle: ???
+  persistent_workers: true
+  collate_fn:
+    _target_: motionbricks.motionlib.data.utils.collate_batch
+    _partial_: true
+    llm_shape: ${model.denoiser.backbone.llm_shape}
+    num_frames: null
+data:
+  _target_: motionbricks.motionlib.data.motion_dataset.MotionDataset
+  name: motionbricks-G1
+  folder: ../datasets/motionbricks-G1
+  text_embeddings: null
+  split: ???
+  motion_sampler:
+    _target_: motionbricks.motionlib.data.motion_sampler.MaxDurationRandomCrop
+    max_seconds: 30
+  motion_loading_mode: memmap
+  use_natural_desc: true
+  use_short_desc: true
+  use_technical_desc: false
+  use_nv_overview_desc: false
+  nv_overview_prob: null
+  drop_text_prob: 0.1
+  augment_text: false
+  aug_text_suffix: ''
+  aug_text_prob: 0.75
+  aug_text_ind_range:
+  - 1
+  - 61
+  load_neutral_joints: false
+  timelines_mode: false
+  timelines_name: null
+  dataset_sampler: null
+  dataset_sampler_only_at_train: true
+  to_canonicalize: false
+  randomize_first_heading: true
+  loading_mode: motion_only
+skeleton:
+  base_name: g1skel34
+  folder: out/motionbricks_root/version_1/skeleton
+  t_pose: capture
+  orig_prefix: ''
+  name: g1skel34
+  _target_: motionbricks.motionlib.core.skeletons.g1.G1Skeleton34
+motion_rep:
+  name: ${skeleton.name}_dual_root_global_joints
+  stats:
+    _target_: motionbricks.motionlib.core.utils.stats.Stats
+    folder: out/motionbricks_root/version_1/stats/motion
+  _target_: motionbricks.motionlib.core.motion_reps.dual_root_global_joints.GlobalRootGlobalJoints
+fps: 30
+wandb_run: motionbricks_root
+resume: false
+version: 1
+cp: last_epoch
+matmul_precision: high
+seed: 7
+id: ${hydra:runtime.choices.exp}_gpu${trainer.devices}_node${trainer.num_nodes}
+out_dir: out/motionbricks_root
+run_dir: ???
+pure_testing: false
+config_version: 4
+data_root: datasets
+tmr_checkpoints_root: null
+use_one_logger: false

g1/out/motionbricks_root/version_1/meta.yaml

@@ -0,0 +1 @@
+wandb_run: motionbricks_root

g1/out/motionbricks_root/version_1/skeleton/joints.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0
+size 944

g1/out/motionbricks_root/version_1/skeleton/parents.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4
+size 400

g1/out/motionbricks_root/version_1/skeleton/skeleton.yaml

@@ -0,0 +1,6 @@
+base_name: g1skel34
+folder: ${data.folder}/features/motion_rep/${motion_rep.name}_fps_${fps}/skeleton
+t_pose: capture
+orig_prefix: ''
+name: g1skel34
+_target_: motionbricks.motionlib.core.skeletons.g1.G1Skeleton34

g1/out/motionbricks_root/version_1/stats/motion/mean.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea
+size 3472

g1/out/motionbricks_root/version_1/stats/motion/std.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7
+size 3472

g1/out/motionbricks_vqvae/version_1/checkpoints/model-step=2000000.json

@@ -0,0 +1,555 @@
+{
+  "schema_version": 1,
+  "source_checkpoint_sha256": "f12a09d46ad390a8e2eecbe7219b2472fcab6b59df0a13f6a40c35cb6da4d99a",
+  "epoch": 25973,
+  "global_step": 2000000,
+  "pytorch_lightning_version": "2.4.0",
+  "state_dict_metadata": {
+    "": {
+      "version": 1
+    },
+    "pose_net": {
+      "version": 1
+    },
+    "pose_net._motion_rep": {
+      "version": 1
+    },
+    "pose_net._motion_rep.skeleton": {
+      "version": 1
+    },
+    "pose_net._motion_rep.stats": {
+      "version": 1
+    },
+    "pose_net.encoder": {
+      "version": 1
+    },
+    "pose_net.encoder.model": {
+      "version": 1
+    },
+    "pose_net.encoder.model.0": {
+      "version": 1
+    },
+    "pose_net.encoder.model.1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.0": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.0": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.0.norm1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.0.norm2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.0.activation1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.0.activation2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.0.conv1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.0.conv2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.1.norm1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.1.norm2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.1.activation1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.1.activation2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.1.conv1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.1.conv2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.2.norm1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.2.norm2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.2.activation1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.2.activation2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.2.conv1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.2.conv2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.3": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.3.norm1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.3.norm2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.3.activation1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.3.activation2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.3.conv1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.2.1.model.3.conv2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.0": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.0": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.0.norm1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.0.norm2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.0.activation1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.0.activation2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.0.conv1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.0.conv2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.1.norm1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.1.norm2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.1.activation1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.1.activation2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.1.conv1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.1.conv2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.2.norm1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.2.norm2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.2.activation1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.2.activation2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.2.conv1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.2.conv2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.3": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.3.norm1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.3.norm2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.3.activation1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.3.activation2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.3.conv1": {
+      "version": 1
+    },
+    "pose_net.encoder.model.3.1.model.3.conv2": {
+      "version": 1
+    },
+    "pose_net.encoder.model.4": {
+      "version": 1
+    },
+    "pose_net.decoder": {
+      "version": 1
+    },
+    "pose_net.decoder.model": {
+      "version": 1
+    },
+    "pose_net.decoder.model.0": {
+      "version": 1
+    },
+    "pose_net.decoder.model.1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.0": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.0.norm1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.0.norm2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.0.activation1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.0.activation2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.0.conv1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.0.conv2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.1.norm1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.1.norm2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.1.activation1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.1.activation2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.1.conv1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.1.conv2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.2.norm1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.2.norm2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.2.activation1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.2.activation2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.2.conv1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.2.conv2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.3": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.3.norm1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.3.norm2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.3.activation1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.3.activation2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.3.conv1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.0.model.3.conv2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.2.2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.0": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.0.norm1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.0.norm2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.0.activation1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.0.activation2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.0.conv1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.0.conv2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.1.norm1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.1.norm2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.1.activation1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.1.activation2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.1.conv1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.1.conv2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.2.norm1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.2.norm2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.2.activation1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.2.activation2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.2.conv1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.2.conv2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.3": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.3.norm1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.3.norm2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.3.activation1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.3.activation2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.3.conv1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.0.model.3.conv2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.1": {
+      "version": 1
+    },
+    "pose_net.decoder.model.3.2": {
+      "version": 1
+    },
+    "pose_net.decoder.model.4": {
+      "version": 1
+    },
+    "pose_net.decoder.model.5": {
+      "version": 1
+    },
+    "pose_net.decoder.model.6": {
+      "version": 1
+    },
+    "pose_net.decoder.external_cond_blocks": {
+      "version": 1
+    },
+    "pose_net.decoder.external_cond_blocks.0": {
+      "version": 1
+    },
+    "pose_net.decoder.external_cond_blocks.1": {
+      "version": 1
+    },
+    "pose_net.decoder.external_cond_blocks.2": {
+      "version": 1
+    },
+    "pose_net.decoder.external_cond_blocks.3": {
+      "version": 1
+    },
+    "pose_net.decoder.target_cond_blocks": {
+      "version": 1
+    },
+    "pose_net.decoder.target_cond_blocks.0": {
+      "version": 1
+    },
+    "pose_net.decoder.target_cond_blocks.1": {
+      "version": 1
+    },
+    "pose_net.decoder.target_cond_blocks.2": {
+      "version": 1
+    },
+    "pose_net.decoder.target_cond_blocks.3": {
+      "version": 1
+    },
+    "pose_net.quantizer": {
+      "version": 1
+    },
+    "pose_net.quantizer.vq": {
+      "version": 1
+    },
+    "pose_net.quantizer.vq.project_in": {
+      "version": 1
+    },
+    "pose_net.quantizer.vq.project_out": {
+      "version": 1
+    },
+    "pose_net.quantizer.vq._codebook": {
+      "version": 1
+    },
+    "motion_rep": {
+      "version": 1
+    },
+    "motion_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.stats": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.global_motion_rep": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.global_motion_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.global_motion_rep.stats": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.local_motion_rep": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.local_motion_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.local_motion_rep.stats": {
+      "version": 1
+    },
+    "motion_rep.stats": {
+      "version": 1
+    },
+    "global_motion_rep": {
+      "version": 1
+    },
+    "global_motion_rep.skeleton": {
+      "version": 1
+    },
+    "global_motion_rep.stats": {
+      "version": 1
+    },
+    "local_motion_rep": {
+      "version": 1
+    },
+    "local_motion_rep.skeleton": {
+      "version": 1
+    },
+    "local_motion_rep.stats": {
+      "version": 1
+    }
+  },
+  "omitted_training_state": [
+    "loops",
+    "callbacks",
+    "optimizer_states",
+    "lr_schedulers"
+  ]
+}

g1/out/motionbricks_vqvae/version_1/checkpoints/model-step=2000000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf9c31173b334303bf392d184c24701ad1a095b3e94a5fe6b11db4d01343a454
+size 95194080

g1/out/motionbricks_vqvae/version_1/config.yaml

@@ -0,0 +1,228 @@
+model:
+  pose_vqvae_network:
+    _target_: motionbricks.vqvae.neural_modules.vqvae.VQVAE
+    pose_root_mode: pose
+    motion_rep: ???
+    encoder_state_dim: 241
+    decoder_state_dim: 329
+    decoder_target_cond_dim: 241
+    decoder_external_cond_dim: 2
+    feature_mode:
+    - joint_positions_and_rotations_and_hip_height
+    - pose
+    - joint_positions_and_rotations_and_hip_height
+    - root_without_hip_height_without_heading
+    quantizer_strategy: multihead_ema_reset
+    quantizer_mu: 0.99
+    nb_code: 100000000
+    code_dim: 256
+    output_emb_width: 256
+    down_t: 2
+    stride_t: 2
+    width: 512
+    depth: 4
+    dilation_growth_rate: 3
+    activation: relu
+    num_heads: 8
+    kmeans_init: false
+    norm: None
+    calculate_per_head_perplexity: true
+    cond_fusion_last_layer: false
+  _target_: motionbricks.vqvae.models.motion_vqvae.MotionVQVAEModel
+  motion_rep: ???
+  pose_vqvae_motion_rep: local
+  root_vqvae_network: null
+  denoiser:
+    backbone:
+      llm_shape: null
+  args:
+    min_tokens: 6
+    max_tokens: 16
+    down_t: 2
+    commit_loss_coeff: 0.02
+    batchsize_mul_factor: 2
+    root_vqvae_max_num_keyframes: 5
+    root_vqvae_no_keyframe_prob: 0.25
+    pose_vqvae_max_num_keyframes: 10
+    pose_vqvae_no_keyframe_prob: 0.25
+    keyframe_num_warmup_steps: 200000
+    global_root_loss_coeff: 0.5
+    local_root_loss_coeff: 0.5
+    max_angle_diff: 5.0
+    max_norm_ratio: 0.2
+    percentage_of_perturbed_samples: 0.2
+    floor_estimation: min_joint_height_within_windows
+    floor_estimation_window_time: 1.0
+    skate_contact_loss_coeff: 0.01
+    joint_vel_loss_coeff: 2.0
+  optimizer:
+    _target_: adam_atan2_pytorch.AdamAtan2
+    _partial_: true
+    lr: 0.0002
+    weight_decay: 0.0
+  scheduler:
+    _target_: motionbricks.motionlib.train.scheduler.WarmupCosineScheduler
+    _partial_: true
+    num_warmup_steps: 10000
+    num_training_steps: ${trainer.max_steps}
+    final_lr: 4.0e-06
+    last_epoch: -1
+    lt_kwargs:
+      interval: step
+      frequency: 1
+trainer:
+  _target_: pytorch_lightning.Trainer
+  precision: 32
+  enable_progress_bar: false
+  profiler: null
+  check_val_every_n_epoch: null
+  detect_anomaly: false
+  num_sanity_val_steps: 0
+  max_steps: 2000001
+  gradient_clip_val: 0.5
+  log_every_n_steps: 50
+  val_check_interval: 50000
+  devices: 8
+  num_nodes: 4
+  accelerator: gpu
+  strategy: ddp
+loggers:
+  out_dir: ${out_dir}
+  cfg_id: ${id}
+  max_steps: ${trainer.max_steps}
+  log_nsteps: ${trainer.log_every_n_steps}
+  use_wandb: true
+  wandb_run: ${wandb_run}
+  wandb_group: null
+  wandb_project: motionbricks
+  wandb_entity: null
+  wandb_run_name: motionbricks_vqvae
+  logger_project: motionbricks
+callbacks:
+  callback_dict:
+    checkpoint_epoch_cb:
+      _target_: motionbricks.motionlib.train.callbacks.ckpt.ReadPermissionModelCheckpoint
+      monitor: null
+      dirpath: out/motionbricks_vqvae/version_1/checkpoints
+      filename: model-{step:07d}
+      save_last: false
+      save_top_k: -1
+      mode: min
+      every_n_train_steps: 50000
+    learning_rate_monitor:
+      _target_: pytorch_lightning.callbacks.LearningRateMonitor
+      logging_interval: step
+    autoresume:
+      _target_: motionbricks.motionlib.train.callbacks.autoresume.AutoResumeCallback
+      autoresume_after: 1260000
+      save_checkpoint_on_exception: true
+    eval_visualize:
+      _target_: motionbricks.vqvae.callbacks.evaluation.VQVAE_Evaluate
+      every_n_steps: 50001
+      train_dataloader: ???
+      val_dataloader: ???
+      seed: 2333
+      batch_size: 32
+      max_num_data: 40000
+      max_num_visualization: 32
+      out_dir: ${run_dir}/vis
+      metrics:
+      - _target_: motionbricks.eval.metrics.FootSkateFromHeight
+      - _target_: motionbricks.eval.metrics.FootSkateFromContacts
+      - _target_: motionbricks.eval.metrics.JointPosConsistency
+    train_end:
+      _target_: motionbricks.motionlib.train.callbacks.train_end.TrainEndCallback
+  only_rank_zero:
+  - eval_visualize
+  - learning_rate_monitor
+  - evaluate
+  - train_end
+  need_version: []
+  need_dataloaders:
+  - eval_visualize
+  - evaluate
+  need_ema: []
+  need_motionbricks_test: []
+  need_log:
+  - eval_visualize
+  - evaluate
+  need_autoresume: []
+  need_mask_cond_sampler: []
+  need_one_logger_callback:
+  - autoresume
+dataloader:
+  _target_: torch.utils.data.DataLoader
+  batch_size: 128
+  num_workers: 11
+  shuffle: ???
+  persistent_workers: true
+  collate_fn:
+    _target_: motionbricks.motionlib.data.utils.collate_batch
+    _partial_: true
+    llm_shape: ${model.denoiser.backbone.llm_shape}
+    num_frames: null
+data:
+  _target_: motionbricks.motionlib.data.motion_dataset.MotionDataset
+  name: motionbricks-G1
+  folder: ../datasets/motionbricks-G1
+  text_embeddings: null
+  split: ???
+  motion_sampler:
+    _target_: motionbricks.motionlib.data.motion_sampler.MaxDurationRandomCrop
+    max_seconds: 30
+  motion_loading_mode: memmap
+  use_natural_desc: true
+  use_short_desc: true
+  use_technical_desc: false
+  use_nv_overview_desc: false
+  nv_overview_prob: null
+  drop_text_prob: 0.1
+  augment_text: false
+  aug_text_suffix: ''
+  aug_text_prob: 0.75
+  aug_text_ind_range:
+  - 1
+  - 61
+  load_neutral_joints: false
+  timelines_mode: false
+  timelines_name: null
+  dataset_sampler: null
+  dataset_sampler_only_at_train: true
+  to_canonicalize: false
+  randomize_first_heading: true
+  loading_mode: motion_only
+skeleton:
+  base_name: g1skel34
+  folder: out/motionbricks_vqvae/version_1/skeleton
+  t_pose: capture
+  orig_prefix: ''
+  name: g1skel34
+  _target_: motionbricks.motionlib.core.skeletons.g1.G1Skeleton34
+motion_rep:
+  name: ${skeleton.name}_dual_root_global_joints
+  stats:
+    _target_: motionbricks.motionlib.core.utils.stats.Stats
+    folder: out/motionbricks_vqvae/version_1/stats/motion
+  _target_: motionbricks.motionlib.core.motion_reps.dual_root_global_joints.GlobalRootGlobalJoints
+fps: 30
+wandb_run: motionbricks_vqvae
+resume: false
+version: 1
+cp: last_epoch
+matmul_precision: high
+seed: 7
+id: ${hydra:runtime.choices.exp}_gpu${trainer.devices}_node${trainer.num_nodes}
+out_dir: out/motionbricks_vqvae
+run_dir: out/motionbricks_vqvae/version_1
+pure_testing: false
+test:
+  unlimit_max_code: false
+  max_num_data: 16
+  min_length: 10
+  max_length: 60
+  ckpt_dir: null
+  ckpt_name: null
+config_version: 4
+data_root: datasets
+tmr_checkpoints_root: null
+use_one_logger: false

g1/out/motionbricks_vqvae/version_1/hparams.yaml

@@ -0,0 +1,228 @@
+model:
+  pose_vqvae_network:
+    _target_: motionbricks.vqvae.neural_modules.vqvae.VQVAE
+    pose_root_mode: pose
+    motion_rep: ???
+    encoder_state_dim: 241
+    decoder_state_dim: 329
+    decoder_target_cond_dim: 241
+    decoder_external_cond_dim: 2
+    feature_mode:
+    - joint_positions_and_rotations_and_hip_height
+    - pose
+    - joint_positions_and_rotations_and_hip_height
+    - root_without_hip_height_without_heading
+    quantizer_strategy: multihead_ema_reset
+    quantizer_mu: 0.99
+    nb_code: 100000000
+    code_dim: 256
+    output_emb_width: 256
+    down_t: 2
+    stride_t: 2
+    width: 512
+    depth: 4
+    dilation_growth_rate: 3
+    activation: relu
+    num_heads: 8
+    kmeans_init: false
+    norm: None
+    calculate_per_head_perplexity: true
+    cond_fusion_last_layer: false
+  _target_: motionbricks.vqvae.models.motion_vqvae.MotionVQVAEModel
+  motion_rep: ???
+  pose_vqvae_motion_rep: local
+  root_vqvae_network: null
+  denoiser:
+    backbone:
+      llm_shape: null
+  args:
+    min_tokens: 6
+    max_tokens: 16
+    down_t: 2
+    commit_loss_coeff: 0.02
+    batchsize_mul_factor: 2
+    root_vqvae_max_num_keyframes: 5
+    root_vqvae_no_keyframe_prob: 0.25
+    pose_vqvae_max_num_keyframes: 10
+    pose_vqvae_no_keyframe_prob: 0.25
+    keyframe_num_warmup_steps: 200000
+    global_root_loss_coeff: 0.5
+    local_root_loss_coeff: 0.5
+    max_angle_diff: 5.0
+    max_norm_ratio: 0.2
+    percentage_of_perturbed_samples: 0.2
+    floor_estimation: min_joint_height_within_windows
+    floor_estimation_window_time: 1.0
+    skate_contact_loss_coeff: 0.01
+    joint_vel_loss_coeff: 2.0
+  optimizer:
+    _target_: adam_atan2_pytorch.AdamAtan2
+    _partial_: true
+    lr: 0.0002
+    weight_decay: 0.0
+  scheduler:
+    _target_: motionbricks.motionlib.train.scheduler.WarmupCosineScheduler
+    _partial_: true
+    num_warmup_steps: 10000
+    num_training_steps: ${trainer.max_steps}
+    final_lr: 4.0e-06
+    last_epoch: -1
+    lt_kwargs:
+      interval: step
+      frequency: 1
+trainer:
+  _target_: pytorch_lightning.Trainer
+  precision: 32
+  enable_progress_bar: false
+  profiler: null
+  check_val_every_n_epoch: null
+  detect_anomaly: false
+  num_sanity_val_steps: 0
+  max_steps: 2000001
+  gradient_clip_val: 0.5
+  log_every_n_steps: 50
+  val_check_interval: 50000
+  devices: 8
+  num_nodes: 4
+  accelerator: gpu
+  strategy: ddp
+loggers:
+  out_dir: ${out_dir}
+  cfg_id: ${id}
+  max_steps: ${trainer.max_steps}
+  log_nsteps: ${trainer.log_every_n_steps}
+  use_wandb: true
+  wandb_run: ${wandb_run}
+  wandb_group: null
+  wandb_project: motionbricks
+  wandb_entity: null
+  wandb_run_name: motionbricks_vqvae
+  logger_project: motionbricks
+callbacks:
+  callback_dict:
+    checkpoint_epoch_cb:
+      _target_: motionbricks.motionlib.train.callbacks.ckpt.ReadPermissionModelCheckpoint
+      monitor: null
+      dirpath: null
+      filename: model-{step:07d}
+      save_last: false
+      save_top_k: -1
+      mode: min
+      every_n_train_steps: 50000
+    learning_rate_monitor:
+      _target_: pytorch_lightning.callbacks.LearningRateMonitor
+      logging_interval: step
+    autoresume:
+      _target_: motionbricks.motionlib.train.callbacks.autoresume.AutoResumeCallback
+      autoresume_after: 1260000
+      save_checkpoint_on_exception: true
+    eval_visualize:
+      _target_: motionbricks.vqvae.callbacks.evaluation.VQVAE_Evaluate
+      every_n_steps: 50001
+      train_dataloader: ???
+      val_dataloader: ???
+      seed: 2333
+      batch_size: 32
+      max_num_data: 40000
+      max_num_visualization: 32
+      out_dir: ${run_dir}/vis
+      metrics:
+      - _target_: motionbricks.eval.metrics.FootSkateFromHeight
+      - _target_: motionbricks.eval.metrics.FootSkateFromContacts
+      - _target_: motionbricks.eval.metrics.JointPosConsistency
+    train_end:
+      _target_: motionbricks.motionlib.train.callbacks.train_end.TrainEndCallback
+  only_rank_zero:
+  - eval_visualize
+  - learning_rate_monitor
+  - evaluate
+  - train_end
+  need_version: []
+  need_dataloaders:
+  - eval_visualize
+  - evaluate
+  need_ema: []
+  need_motionbricks_test: []
+  need_log:
+  - eval_visualize
+  - evaluate
+  need_autoresume: []
+  need_mask_cond_sampler: []
+  need_one_logger_callback:
+  - autoresume
+dataloader:
+  _target_: torch.utils.data.DataLoader
+  batch_size: 128
+  num_workers: 11
+  shuffle: ???
+  persistent_workers: true
+  collate_fn:
+    _target_: motionbricks.motionlib.data.utils.collate_batch
+    _partial_: true
+    llm_shape: ${model.denoiser.backbone.llm_shape}
+    num_frames: null
+data:
+  _target_: motionbricks.motionlib.data.motion_dataset.MotionDataset
+  name: motionbricks-G1
+  folder: ../datasets/motionbricks-G1
+  text_embeddings: null
+  split: ???
+  motion_sampler:
+    _target_: motionbricks.motionlib.data.motion_sampler.MaxDurationRandomCrop
+    max_seconds: 30
+  motion_loading_mode: memmap
+  use_natural_desc: true
+  use_short_desc: true
+  use_technical_desc: false
+  use_nv_overview_desc: false
+  nv_overview_prob: null
+  drop_text_prob: 0.1
+  augment_text: false
+  aug_text_suffix: ''
+  aug_text_prob: 0.75
+  aug_text_ind_range:
+  - 1
+  - 61
+  load_neutral_joints: false
+  timelines_mode: false
+  timelines_name: null
+  dataset_sampler: null
+  dataset_sampler_only_at_train: true
+  to_canonicalize: false
+  randomize_first_heading: true
+  loading_mode: motion_only
+skeleton:
+  base_name: g1skel34
+  folder: out/motionbricks_vqvae/version_1/skeleton
+  t_pose: capture
+  orig_prefix: ''
+  name: g1skel34
+  _target_: motionbricks.motionlib.core.skeletons.g1.G1Skeleton34
+motion_rep:
+  name: ${skeleton.name}_dual_root_global_joints
+  stats:
+    _target_: motionbricks.motionlib.core.utils.stats.Stats
+    folder: out/motionbricks_vqvae/version_1/stats/motion
+  _target_: motionbricks.motionlib.core.motion_reps.dual_root_global_joints.GlobalRootGlobalJoints
+fps: 30
+wandb_run: motionbricks_vqvae
+resume: false
+version: 1
+cp: last_epoch
+matmul_precision: high
+seed: 7
+id: ${hydra:runtime.choices.exp}_gpu${trainer.devices}_node${trainer.num_nodes}
+out_dir: out/motionbricks_vqvae
+run_dir: ???
+pure_testing: false
+test:
+  unlimit_max_code: false
+  max_num_data: 16
+  min_length: 10
+  max_length: 60
+  ckpt_dir: null
+  ckpt_name: null
+config_version: 4
+data_root: datasets
+tmr_checkpoints_root: null
+use_one_logger: false

g1/out/motionbricks_vqvae/version_1/meta.yaml

@@ -0,0 +1 @@
+wandb_run: motionbricks_vqvae

g1/out/motionbricks_vqvae/version_1/skeleton/joints.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a682fc9d36204eb8e58c6b29eb0881e36242ed94dff078c82a373cdc6783ab0
+size 944

g1/out/motionbricks_vqvae/version_1/skeleton/parents.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02754fe01a8dcad13b9e3f95f7a696f56bccc5eeb5f5bf701f31cec88e23b1c4
+size 400

g1/out/motionbricks_vqvae/version_1/skeleton/skeleton.yaml

@@ -0,0 +1,6 @@
+base_name: g1skel34
+folder: ${data.folder}/features/motion_rep/${motion_rep.name}_fps_${fps}/skeleton
+t_pose: capture
+orig_prefix: ''
+name: g1skel34
+_target_: motionbricks.motionlib.core.skeletons.g1.G1Skeleton34

g1/out/motionbricks_vqvae/version_1/stats/motion/mean.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca390f0081e2373ab71e860a3546cb70cc11fdfac6ce525155403e047b66fdea
+size 3472

g1/out/motionbricks_vqvae/version_1/stats/motion/std.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fca7dd6135cbe96504b1307a7db97e9004556937a2fc72d139077b962cee6bd7
+size 3472

soma/README.md

@@ -0,0 +1,374 @@
+---
+library_name: pytorch
+tags:
+  - nvidia
+  - motionbricks
+  - soma
+  - rigplay
+  - motion-generation
+  - pose-conditioned
+  - animation
+---
+
+# MotionBricks SOMA
+
+[Project page](https://nvlabs.github.io/motionbricks/) · [Paper](https://arxiv.org/abs/2604.24833)
+
+## Model Description
+
+MotionBricks SOMA generates human skeletal motion from recent motion, target poses, and
+root-motion constraints. Applications can feed generated motion back into the model and update
+targets to produce continuous locomotion and transitions between authored poses.
+
+NVIDIA developed `MotionBricks-SOMA-RP-v1` for the 30-joint SOMA skeleton using Bones RigPlay 1
+(RP1) motion data. The checkpoint contains a pose tokenizer and decoder, a pose predictor, and a
+root predictor. Together, these components generate skeletal motion from the supplied constraints.
+
+The package includes single-precision (FP32) weights in SafeTensors format, configuration,
+normalization statistics, and skeleton metadata. A training pipeline is not included.
+
+This model is intended for commercial and non-commercial motion-generation applications.
+
+### Where to Start
+
+To run MotionBricks in an application, start with the [SOMA ONNX model and runtime](https://github.com/NVlabs/motionbricks/blob/main/docs/soma_model.md).
+Use the checkpoints on this page to work with the individual PyTorch components or create
+ONNX exports. [Download the complete SOMA package](#download-and-files), then follow the
+[exporter guide](https://github.com/NVlabs/motionbricks/blob/main/tools/onnx/README.md).
+
+## License and Terms of Use
+
+Use of this model is governed by the [NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/).
+It permits commercial use and redistribution of modified models, subject to the agreement's terms.
+
+- The accompanying software uses [Apache 2.0](https://github.com/NVlabs/motionbricks/blob/main/LICENSE).
+- The listed Bones-derived human BVH and G1 CSV samples use the
+  [NVIDIA Sample Data Evaluation License](https://github.com/NVlabs/motionbricks/blob/main/assets/motions/LICENSE.txt).
+  They are for internal evaluation and testing of NVIDIA technologies. See the
+  [sample notice](https://github.com/NVlabs/motionbricks/blob/main/assets/motions/README.md) for the exact files covered.
+- Robot assets and third-party dependencies retain their [respective licenses](https://github.com/NVlabs/motionbricks/tree/main/licenses).
+
+## Deployment Geography
+
+Global.
+
+## Intended Uses
+
+MotionBricks SOMA is intended for developers and researchers building interactive human animation
+and motion previews for robotics and simulation. Applications can generate transitions between
+authored poses or update targets to guide locomotion. Generated motion is not validated for
+physical robot control.
+
+## Model Package
+
+The SOMA model package contains SafeTensors weights, their supporting metadata, and this model card.
+Derived Open Neural Network Exchange (ONNX) exports and runtime software are maintained in the
+MotionBricks software repository.
+
+The checkpoint package is organized under [`soma/` in the MotionBricks model repository](https://huggingface.co/nvidia/MotionBricks/tree/main/soma). The three SOMA components are used together. The separate `g1/` package is not covered by this card.
+
+This card identifies the checkpoint as `MotionBricks-SOMA-RP-v1`. The accompanying software
+release is v0.2.0. The checkpoint name and software version describe different artifacts.
+
+## References
+
+T. Wang, O. Dionne, M. De Ruyter, D. Minor, D. Rempe, K. Zhao, M. Petrovich, Y. Yuan, C. Li, Z. Luo, B. Robison, X. Blackwell, B. Antoniazzi, X. B. Peng, Y. Zhu and S. Yuen. **MotionBricks: Scalable Real-Time Motions with Modular Latent Generative Model and Smart Primitives**, 2026. [ACM Transactions on Graphics](https://doi.org/10.1145/3811334). [arXiv:2604.24833](https://arxiv.org/abs/2604.24833).
+
+## Download and Files
+
+Download the complete [`soma/` folder](https://huggingface.co/nvidia/MotionBricks/tree/main/soma)
+and keep its directory structure intact. It contains separate directories for the pose
+vector-quantized variational autoencoder (VQ-VAE), pose predictor, and root predictor.
+No archive extraction is needed.
+
+Each component includes `checkpoints/model-step=1500000.safetensors`, a JSON metadata sidecar,
+and supporting configuration files. Normalization arrays and skeleton metadata define the
+motion representation. Joint data is stored in `joints.npy`. Keep these files with the weights.
+
+Use the [manifest](https://huggingface.co/nvidia/MotionBricks/blob/main/soma/manifest.json) to
+identify package files and the [checksums](https://huggingface.co/nvidia/MotionBricks/blob/main/soma/SHA256SUMS)
+to verify their integrity.
+
+The package omits optimizer, scheduler, callback, and training-loop state. It provides weights
+for inference or fine-tuning initialization with a compatible implementation, not exact
+resumption of the original training run.
+
+| Property | Value |
+| --- | --- |
+| Model family | MotionBricks |
+| Task | Pose-conditioned motion generation and inbetweening |
+| Skeleton | SOMA, 30 joints |
+| Training dataset | RP1 |
+| FPS | 30 |
+| Checkpoint components | Pose VQ-VAE, pose masked motion model, root masked motion model |
+| Checkpoint step | 1,500,000 for each component |
+| Training precision | FP32 |
+| Weight format | SafeTensors, `.safetensors` |
+
+## Model Architecture
+
+**Architecture Type:** Modular latent generative model.
+
+**Network Architecture:** A pose vector-quantized variational autoencoder (VQ-VAE) encodes and decodes motion tokens. One transformer predicts motion tokens. Another predicts root motion and duration. The SOMA tokenizer uses five heads with 16 codes per head and 32 dimensions per head.
+
+**Base Model:** The SOMA components were trained from scratch on RP1 motion. They are distinct from the original G1 model weights.
+
+**Number of Model Parameters:** Approximately 200 million across the three components. The component counts below are rounded.
+
+| Component | Approximate parameters |
+| --- | ---: |
+| Pose VQ-VAE | 23.4 million |
+| Pose predictor | 136.5 million |
+| Root predictor | 40.4 million |
+
+Smart Primitives and locomotion controllers supply constraints at the application level. Text prompts are outside this model's conditioning interface. See the [MotionBricks paper](https://arxiv.org/abs/2604.24833) for the method. The paper reports experiments with the model configurations and datasets specified there.
+
+## Inputs
+
+**Input Type(s):** Skeletal-motion features, pose constraints, root-motion constraints, motion-token IDs, conditioning masks and duration constraints. There are no text, image, video or scene-geometry inputs in these checkpoint interfaces.
+
+**Input Format(s):** Numeric tensors consumed by the matching PyTorch implementation. Motion features are FP32 with bundled normalization statistics. Token IDs and lengths are integers. Dense constraint and padding masks are Boolean.
+
+**Input Parameters:** Component-specific batched tensor shapes are listed below. `N` is batch size, `L` is the number of motion-token positions and `T = 4L` is the number of frames.
+
+**Other Properties Related to Input:** Generation uses 6 to 16 tokens, corresponding to 24 to 64 frames at 30 Hz. Shorter sequences can be padded with valid-length masks. The root predictor uses eight boundary frames, four context frames and four target frames. Practical batch sizes depend on memory and implementation. These are checkpoint-component interfaces after feature preparation. An application's pose interface or exported graph can differ.
+
+For joint ordering, coordinates, normalization and reconstruction, use the matching MotionBricks implementation and package metadata.
+
+### Pose VQ-VAE: tokenizer and decoder
+
+| Tensor | Direction | Shape | Description |
+| --- | --- | --- | --- |
+| Pose features | Encoder input | `(N, T, 268)` | Root height, 29 non-root joint positions with three values each, and 30 joint rotations with six values each |
+| Motion codes | Encoder output / decoder input | `(N, L, 5)` | Five codebook heads, each with IDs from 0 through 15 |
+| Target pose features | Decoder input | `(N, T, 268)` | Pose constraints in the same representation |
+| Target pose mask | Decoder input | `(N, T)` | Frames containing pose constraints |
+| Local root velocity | Decoder input | `(N, T, 2)` | Ground root-motion conditioning |
+| Valid-token mask | Decoder input, optional | `(N, L)` | Valid positions for padded sequences |
+| Reconstructed motion | Decoder output | `(N, T, 365)` | Four local-root features, 87 joint-position values, 180 rotation values, 90 joint-velocity values and four foot-contact features |
+
+Each token represents four frames. The quantized latent has 160 values per token, five heads times 32 values. The table uses separate per-head code IDs. The implementation also supports packing them into one integer per token.
+
+### Pose predictor
+
+| Tensor | Direction | Shape | Description |
+| --- | --- | --- | --- |
+| Motion tokens | Input | `(N, L, 5)` | IDs from 0 through 15, or mask ID 16 for unknown codes |
+| Root trajectory | Input | `(N, T, 4)` | Ground root position and two-component heading, without root height |
+| Pose constraints | Input | `(N, T, 268)` | Root height, joint positions and 6D rotations |
+| Pose constraint mask | Input | `(N, T)` | Frames with supplied pose constraints |
+| Valid token count | Input | `(N, 1)` | One length from 6 to 16 per sample |
+| Pose-code logits | Output | `(N, L, 5, 16)` | Scores over each head's codebook. Selected codes feed the decoder. |
+
+### Root predictor
+
+Each sample can have its own constraints and duration.
+
+| Tensor | Direction | Shape | Description |
+| --- | --- | --- | --- |
+| Global root constraints | Input | `(N, 8, 5)` | Root position with three values and heading with two values |
+| Local root constraints | Input | `(N, 8, 4)` | Root angular velocity, ground velocity and height |
+| Pose constraints | Input | `(N, 8, 268)` | Root height, joint positions and 6D rotations |
+| Constraint masks | Input | Three tensors of `(N, 8)` | One mask for each constraint tensor above |
+| Requested token count | Input | `(N, 1)` | Duration from 6 to 16, or unknown-duration token 18 |
+| Allowed durations | Input, optional | `(N, 11)` | Flags for token counts from 6 to 16 |
+| Duration logits | Output | `(N, 12)` | Eleven duration scores and an out-of-reach class |
+| Predicted token count | Output | `(N,)` | Selected duration. Generation disables the out-of-reach class. |
+| Predicted global root | Output | `(N, 64, 5)` | Padded root trajectory. Valid length is four times each sample's token count. |
+
+## Outputs
+
+**Output Type(s):** Motion codes and logits, reconstructed skeletal-motion features, root-motion predictions and valid duration. See the component tables above.
+
+**Output Format(s):** Numeric tensors. Predicted motion features and root motion are FP32. Code IDs and selected lengths are integers. After denormalization and skeletal reconstruction, outputs can be represented as root translations `(N, T, 3)` and joint rotation matrices `(N, T, 30, 3, 3)`.
+
+**Output Parameters:** The component tables specify dimensions and valid lengths. Output motion is represented at 30 Hz. Respect valid-length information rather than treating padded frames as generated content.
+
+**Other Properties Related to Output:** Longer motion requires repeated generation. Predictions may become context for a later call. Export precision and backend can change results. The checkpoint interfaces are not the same as the derived ONNX interfaces.
+
+The model was trained using NVIDIA GPUs. The CPU-based export utility converts these checkpoints
+to ONNX. Its successful execution is not a real-time inference benchmark or a CPU-versus-GPU
+performance comparison.
+
+## Software Integration
+
+**Runtime Engine(s):** PyTorch with the matching MotionBricks implementation, configuration,
+normalization statistics, and skeleton metadata. The exporter requires Python 3.12,
+PyTorch 2.14.0+cpu, NumPy 1.26.4, ONNX 1.23.2, and SafeTensors 0.8.0.
+
+**Supported Hardware Microarchitecture Compatibility:** The exporter runs on the CPU.
+A specific CPU model, minimum-memory requirement, and broader compatibility matrix are
+not documented. For ONNX runtime requirements, see the [SOMA ONNX card](https://github.com/NVlabs/motionbricks/blob/main/docs/soma_model.md).
+
+**Supported Operating System(s):** We tested checkpoint-to-ONNX conversion on Windows.
+Tests on other operating systems are not reported.
+
+The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
+
+The [exporter](https://github.com/NVlabs/motionbricks/blob/main/tools/onnx/README.md) accepts the downloaded checkpoint folder.
+Its loader checks required file sizes and SHA-256 hashes, reads SafeTensors weights,
+and disables NumPy pickle loading. Keep the checkpoint and exporter versions matched.
+Do not load untrusted checkpoints. These checks do not replace security scans or
+validation in your application.
+
+## Release Date
+
+Hugging Face 10/23/2026 via the [MotionBricks SOMA checkpoint package](https://huggingface.co/nvidia/MotionBricks/tree/main/soma).
+
+## Model Versions
+
+`MotionBricks-SOMA-RP-v1` contains the three FP32 components at training step 1,500,000.
+The [manifest](https://huggingface.co/nvidia/MotionBricks/blob/main/soma/manifest.json) and
+[checksums](https://huggingface.co/nvidia/MotionBricks/blob/main/soma/SHA256SUMS) identify the package files.
+
+The components are used together. The MotionBricks exporter combines them into FP32 or
+mixed FP16 ONNX models. Those models have a [separate card](https://github.com/NVlabs/motionbricks/blob/main/docs/soma_model.md) and tensor
+interface. Use that interface when integrating the ONNX runtime. The checkpoint-component
+tensors are not interchangeable with the ONNX interface.
+
+## Training, Testing and Evaluation Datasets
+
+### Training Dataset
+
+**Name/Identifier:** Bones RigPlay 1 (RP1).
+
+**Data Modality:** Other: human skeletal motion capture, represented on the 30-joint SOMA skeleton at 30 Hz. Training used motion features without text embeddings.
+
+**Training Data Size:** 313,526 human-motion clips in the selected training subset. The source
+corpus contains approximately 350,000 clips and 700 hours of motion. The source-corpus duration
+is not the selected training-subset duration.
+
+**Data Collection Method:** Automatic/Sensors. Human performances were recorded in a motion-capture studio by Bones, then provided as skeletal motion. Training uses the processed uniform SOMA representation, not data captured directly on the SOMA skeleton.
+
+**Labeling Method:** Automatic/Sensors for the captured skeletal motion, followed by automated preparation into uniform skeletal features. These motion-only runs use pose/motion values as their learning data and do not consume text embeddings or text annotations. The source dataset can include additional annotations, but those are not model conditioning inputs.
+
+**Properties:** Training uses uniform SOMA motion features at 30 Hz, random first headings, and
+random crops of up to 15 seconds. Root motion and joint features provide the motion representation.
+Text embeddings and text augmentation are disabled. This package does not redistribute the
+training corpus.
+
+**Linguistic Characteristics:** Not applicable to the motion-feature inputs. Associated text
+descriptions are dataset metadata, not model conditioning inputs.
+
+All three checkpoint components were trained in FP32.
+
+### Testing Dataset
+
+**Name, Size and Role:** We used 313,526 clips for training and a separate, selected set of
+4,881 clips for validation and evaluation during training. No clips are shared between these
+sets. The 4,881 clips are a selection from the held-out pool, not the entire pool. A separate
+untouched final-test set, a performer-disjoint split, and the selected subset's duration are
+not documented.
+
+**Data Collection Method:** Automatic/Sensors, from the same Bones studio motion-capture source and processed SOMA representation as the training set.
+
+**Labeling Method:** Automatic/Sensors for captured skeletal motion, followed by automated preparation into the same motion features used for training. Text annotations are not consumed by the recorded motion-only validation path.
+
+**Properties:** The validation inputs are 30 Hz skeletal-motion features in the same uniform SOMA
+representation as the training inputs. Linguistic characteristics are not applicable to these
+motion-only inputs. Disjoint clips alone do not rule out other forms of data leakage.
+
+### Evaluation Dataset and Results
+
+**Name/Size:** Evaluation uses the same 4,881-clip RP1 subset described under
+[Testing Dataset](#testing-dataset). It is not an additional independent test set.
+
+**Benchmark Score:** A complete end-to-end benchmark of these SOMA checkpoints is not included
+with this package.
+
+**Data Collection Method:** Automatic/Sensors. Same studio-captured Bones motion source, processed into the uniform SOMA representation.
+
+**Labeling Method:** Automatic/Sensors for captured skeletal motion, with automated preparation into uniform motion features. Evaluation uses motion rather than text-conditioned labels.
+
+**Properties:** Human skeletal motion represented on SOMA at 30 Hz. Linguistic characteristics
+are not applicable to these motion-only inputs. Text annotations are not consumed. The selected
+subset's duration is unspecified.
+
+**Recorded Results:** The available evidence covers training-time observations and checkpoint
+conversion, not a complete final-checkpoint benchmark.
+
+| Check | Available evidence |
+| --- | --- |
+| Tokenizer | Reconstruction and foot-skating metrics recorded during training |
+| Root predictor | Validation recorded during training |
+| Pose predictor | Training loss recorded, without a complete validation result |
+| Original checkpoints | Validated and exported in July 2026 |
+| SafeTensors conversion | Tested on Windows with CPU PyTorch. See [Inference](#inference) for the environment. |
+
+The [MotionBricks paper](https://arxiv.org/abs/2604.24833) reports method-level experiments with
+the model and dataset configurations specified there. Those results are not an end-to-end
+benchmark of this package.
+
+## Inference
+
+**Acceleration Engine:** The checkpoint implementation and exporter use PyTorch.
+The exporter runs on the CPU. The separate ONNX runtime uses TensorRT 10.8.
+
+**Test Hardware Set-Up:** We tested checkpoint-to-ONNX conversion on Windows using CPU PyTorch.
+The CPU model, conversion time, and checkpoint-runtime latency and throughput are not reported.
+The [ONNX card](https://github.com/NVlabs/motionbricks/blob/main/docs/soma_model.md) describes the separate Windows and RTX 4090 runtime.
+Neither successful export nor runtime execution establishes safe physical robot tracking.
+
+## Limitations
+
+- Output is specific to the SOMA joint layout and proportions.
+- Longer motion requires repeated generation from a short context window.
+- Targets guide motion, but the model does not receive scene geometry. Applications handle obstacles and target placement.
+- Generated motion can contain pose or contact artifacts and vary with numerical precision and inference backend.
+- Kinematic motion generation does not establish physical feasibility or successful tracking by a robot controller.
+
+## Ethical Considerations
+
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
+
+For more detailed information on ethical considerations for this model, see the [Bias](#bias-subcard), [Explainability](#explainability-subcard), [Safety & Security](#safety-and-security-subcard) and [Privacy](#privacy-subcard) subcards below.
+
+Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
+
+## Bias Subcard
+
+| Field | Response |
+| --- | --- |
+| Participation considerations from adversely impacted groups in model design and testing | Not reported. |
+| Measures taken to mitigate unwanted bias | Specific measured mitigation is not reported. Applications should evaluate motion quality for their users and motion conditions. |
+| Bias Metric, if measured | Demographic and body/motion coverage comparisons are not reported. |
+
+The fixed SOMA representation does not establish equivalent quality across body proportions or motion categories.
+
+## Explainability Subcard
+
+| Field | Response |
+| --- | --- |
+| Intended Task/Domain | Pose-conditioned human skeletal motion generation and inbetweening. |
+| Model Type | Modular latent generative model, with transformer predictors and learned pose tokenizer and decoder. |
+| Intended Users | Developers and researchers building animation, robotics, and simulation workflows. |
+| Output | Motion codes and logits, root predictions, duration and reconstructed skeletal-motion features, as detailed in the component tables. |
+| Describe how the model works | The tokenizer represents motion as discrete codes. Pose and root predictors use recent motion and target constraints to predict a transition. The decoder and matching reconstruction produce skeletal motion. Applications can reuse the output as context. |
+| Groups tested to produce comparable outcomes | Demographic subgroup comparisons are not reported. |
+| Technical Limitations and Mitigation | Fixed skeleton, short context, no scene-geometry input and backend/precision variation. Validate application-specific motion, target placement and any physical execution separately. |
+| Verified to have met prescribed NVIDIA quality standards | Yes |
+| Performance Metrics | See [Evaluation Dataset and Results](#evaluation-dataset-and-results) for training metrics and [Inference](#inference) for runtime evidence. |
+| Potential Known Risks | Pose/contact artifacts, unwanted motion under out-of-distribution constraints, and unsafe physical interpretation if generated poses are treated as validated robot commands. |
+| Licensing | Use of this model is governed by the [NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/). See [License and Terms of Use](#license-and-terms-of-use) for separate software and sample-data terms. |
+
+## Privacy Subcard
+
+Training used RP1 skeletal motion collected by Bones and supplied to NVIDIA under a negotiated data license. NVIDIA processed the motion into the SOMA representation.
+
+The datasets used to train this model were reviewed before release. Training used motion features, not text annotations or interactions with this model.
+
+The package does not include the training corpus. Local execution does not eliminate privacy risks associated with training data.
+
+Please review [NVIDIA’s Privacy Policy](https://www.nvidia.com/en-us/about-nvidia/privacy-policy/) for more information.
+
+## Safety and Security Subcard
+
+| Field | Response |
+| --- | --- |
+| Model Application Fields | Industrial/Machinery and Robotics, Media & Entertainment. Generated motion is not validated for physical robot control. |
+| Life-critical impact | Generated skeletal motion is not a validated physical controller. Misuse as robot commands can create physical risk. |
+| Use Case Restrictions | Use of this model is governed by the [NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/). See [License and Terms of Use](#license-and-terms-of-use) for separate software and sample-data terms. |
+| Model and dataset restrictions | This package does not include the RP1 training corpus. Access to the dataset is governed separately from the model weights. |
+
+Evaluate generated motion in the intended application. Physical execution requires a separate tracking and safety system. Hash verification and checkpoint loading alone do not establish complete security or safe downstream behavior.
+

soma/SHA256SUMS

@@ -0,0 +1,24 @@
+50ac7be9443f1bd677a648c610f29b8dd9c6917dddc95a89f7bdbad410638744  README.md
+9ae6cd2d4386b5178e62d47f03f7a1750326a729b919f545fe5cd4a4b966bff0  manifest.json
+d7ead5ef043a9d1e99071b6f8aedc70753560fdc0a613cc1287f1561b50fce3c  soma-v6-nova-4n8g-ea43dd8-fp32-v1/SHA256SUMS
+4ad901e6eb008f40bb5eae22edf25a9d42b3ac553e0c55657a63a0e218d8d015  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json
+8537d9d33a880ff1df6c16c28ed061d60c461a5918d70b1cf99251ff91a10a10  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors
+c2e3b54d5c6aaf0fbcd1e57b61c9306991bdaf841a609c088944c4a460012885  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml
+57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy
+8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml
+802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy
+1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy
+70571a550fda3cfa9fdb1bd3fed502dde27e1f3bcac58546af54a4e8e14705a3  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json
+a48d48c3d8ff98c825ca315797644789ae37f2db9069779a3118f42e84f4f29e  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors
+6ef85c1296e5ba860f79920222684383aa76573b9dbf3299378f05eaf7971d56  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml
+57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy
+8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml
+802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy
+1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6  soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy
+f3518cde99c51f5177a08c1226a45d9992d23e9bb985e658847957cc287a9d00  soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json
+ba0d1db48ba7182ef5d1a72268bfaa1a196f1f1fbf00144c046d947b90684a1a  soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors
+826e58115a0baef638f5a4e42f477d12776a82ac6f0dcda16bc6b8454a88342e  soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml
+57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac  soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy
+8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91  soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml
+802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d  soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy
+1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6  soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy

soma/manifest.json

@@ -0,0 +1,136 @@
+{
+  "schema_version": 2,
+  "model_family": "MotionBricks",
+  "model_variant": "SOMA, 30 joints",
+  "checkpoint_step": 1500000,
+  "weight_format": "safetensors",
+  "weights_only": true,
+  "archive": {
+    "filename": "motionbricks_soma_checkpoints.tar.gz",
+    "bytes": 744243657,
+    "sha256": "290dbdbf16bae9b372d63359d8c3637c40920dfc7f05994446a140a2a4aeb127",
+    "root_directory": "soma-v6-nova-4n8g-ea43dd8-fp32-v1"
+  },
+  "source_archive": {
+    "filename": "motionbricks_soma_checkpoints.tar.gz",
+    "bytes": 2193233205,
+    "sha256": "98f78fa9c46fd7bcd3a28dc00947f0cfb27bb114b011096fd1b11d1c001ba593",
+    "root_directory": "soma-v6-nova-4n8g-ea43dd8-fp32-v1"
+  },
+  "original_checkpoint_bytes_preserved": false,
+  "original_model_tensor_bytes_preserved": true,
+  "skeleton_tensor_bytes_preserved": true,
+  "training_resume_state_included": false,
+  "files": [
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json",
+      "bytes": 16954,
+      "sha256": "4ad901e6eb008f40bb5eae22edf25a9d42b3ac553e0c55657a63a0e218d8d015"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors",
+      "bytes": 546109841,
+      "sha256": "8537d9d33a880ff1df6c16c28ed061d60c461a5918d70b1cf99251ff91a10a10"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml",
+      "bytes": 7751,
+      "sha256": "c2e3b54d5c6aaf0fbcd1e57b61c9306991bdaf841a609c088944c4a460012885"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy",
+      "bytes": 848,
+      "sha256": "57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml",
+      "bytes": 237,
+      "sha256": "8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy",
+      "bytes": 3088,
+      "sha256": "802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy",
+      "bytes": 3088,
+      "sha256": "1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json",
+      "bytes": 18271,
+      "sha256": "70571a550fda3cfa9fdb1bd3fed502dde27e1f3bcac58546af54a4e8e14705a3"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors",
+      "bytes": 161695084,
+      "sha256": "a48d48c3d8ff98c825ca315797644789ae37f2db9069779a3118f42e84f4f29e"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml",
+      "bytes": 6810,
+      "sha256": "6ef85c1296e5ba860f79920222684383aa76573b9dbf3299378f05eaf7971d56"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy",
+      "bytes": 848,
+      "sha256": "57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml",
+      "bytes": 237,
+      "sha256": "8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy",
+      "bytes": 3088,
+      "sha256": "802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy",
+      "bytes": 3088,
+      "sha256": "1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/SHA256SUMS",
+      "bytes": 3009,
+      "sha256": "d7ead5ef043a9d1e99071b6f8aedc70753560fdc0a613cc1287f1561b50fce3c"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json",
+      "bytes": 13245,
+      "sha256": "f3518cde99c51f5177a08c1226a45d9992d23e9bb985e658847957cc287a9d00"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors",
+      "bytes": 93442464,
+      "sha256": "ba0d1db48ba7182ef5d1a72268bfaa1a196f1f1fbf00144c046d947b90684a1a"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml",
+      "bytes": 6654,
+      "sha256": "826e58115a0baef638f5a4e42f477d12776a82ac6f0dcda16bc6b8454a88342e"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy",
+      "bytes": 848,
+      "sha256": "57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml",
+      "bytes": 237,
+      "sha256": "8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy",
+      "bytes": 3088,
+      "sha256": "802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d"
+    },
+    {
+      "path": "soma-v6-nova-4n8g-ea43dd8-fp32-v1/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy",
+      "bytes": 3088,
+      "sha256": "1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6"
+    }
+  ]
+}

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/SHA256SUMS

@@ -0,0 +1,21 @@
+4ad901e6eb008f40bb5eae22edf25a9d42b3ac553e0c55657a63a0e218d8d015  mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json
+8537d9d33a880ff1df6c16c28ed061d60c461a5918d70b1cf99251ff91a10a10  mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors
+c2e3b54d5c6aaf0fbcd1e57b61c9306991bdaf841a609c088944c4a460012885  mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml
+57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac  mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy
+8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91  mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml
+802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d  mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy
+1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6  mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy
+70571a550fda3cfa9fdb1bd3fed502dde27e1f3bcac58546af54a4e8e14705a3  mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json
+a48d48c3d8ff98c825ca315797644789ae37f2db9069779a3118f42e84f4f29e  mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors
+6ef85c1296e5ba860f79920222684383aa76573b9dbf3299378f05eaf7971d56  mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml
+57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac  mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy
+8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91  mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml
+802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d  mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy
+1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6  mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy
+f3518cde99c51f5177a08c1226a45d9992d23e9bb985e658847957cc287a9d00  vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json
+ba0d1db48ba7182ef5d1a72268bfaa1a196f1f1fbf00144c046d947b90684a1a  vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors
+826e58115a0baef638f5a4e42f477d12776a82ac6f0dcda16bc6b8454a88342e  vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml
+57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac  vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy
+8c3ae89053844b00de3a011cad142b57ceed727a7662e51ad1fd4878a1693a91  vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml
+802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d  vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy
+1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6  vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json

@@ -0,0 +1,669 @@
+{
+  "schema_version": 1,
+  "source_checkpoint_sha256": "e12bc7ae9148a817edd45023815da780b96212871023b2cf98ca36d90ef2da99",
+  "epoch": 19480,
+  "global_step": 1500000,
+  "pytorch_lightning_version": "2.4.0",
+  "state_dict_metadata": {
+    "": {
+      "version": 1
+    },
+    "mmm_net": {
+      "version": 1
+    },
+    "mmm_net.motion_rep": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.stats": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.global_motion_rep": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.global_motion_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.global_motion_rep.stats": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.local_motion_rep": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.local_motion_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.local_motion_rep.stats": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.stats": {
+      "version": 1
+    },
+    "mmm_net.global_motion_rep": {
+      "version": 1
+    },
+    "mmm_net.global_motion_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.global_motion_rep.stats": {
+      "version": 1
+    },
+    "mmm_net.local_motion_rep": {
+      "version": 1
+    },
+    "mmm_net.local_motion_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.local_motion_rep.stats": {
+      "version": 1
+    },
+    "mmm_net._transformer_model": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.0": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.0.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.0.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.0.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.0.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.0.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.0.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.0.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.0.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.0.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.1.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.1.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.1.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.1.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.1.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.1.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.1.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.1.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.1.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.2.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.2.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.2.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.2.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.2.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.2.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.2.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.2.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.2.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.3": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.3.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.3.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.3.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.3.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.3.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.3.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.3.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.3.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.3.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.4": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.4.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.4.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.4.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.4.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.4.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.4.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.4.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.4.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.4.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.5": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.5.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.5.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.5.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.5.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.5.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.5.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.5.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.5.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.5.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.6": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.6.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.6.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.6.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.6.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.6.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.6.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.6.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.6.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.6.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.7": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.7.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.7.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.7.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.7.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.7.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.7.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.7.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.7.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.7.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.8": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.8.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.8.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.8.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.8.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.8.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.8.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.8.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.8.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.8.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.9": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.9.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.9.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.9.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.9.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.9.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.9.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.9.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.9.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.9.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.10": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.10.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.10.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.10.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.10.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.10.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.10.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.10.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.10.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.10.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.11": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.11.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.11.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.11.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.11.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.11.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.11.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.11.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.11.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.11.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.12": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.12.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.12.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.12.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.12.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.12.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.12.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.12.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.12.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.12.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.13": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.13.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.13.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.13.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.13.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.13.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.13.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.13.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.13.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.13.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.14": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.14.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.14.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.14.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.14.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.14.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.14.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.14.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.14.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.14.dropout2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.15": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.15.self_attn": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.15.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.15.linear1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.15.dropout": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.15.linear2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.15.norm1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.15.norm2": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.15.dropout1": {
+      "version": 1
+    },
+    "mmm_net._transformer_model.layers.15.dropout2": {
+      "version": 1
+    },
+    "mmm_net._pose_token_emb": {
+      "version": 1
+    },
+    "mmm_net._proj_pose_token_emb": {
+      "version": 1
+    },
+    "mmm_net._proj_pose_token_emb.forward_projection": {
+      "version": 1
+    },
+    "mmm_net._proj_pose_token_emb.fc_layers": {
+      "version": 1
+    },
+    "mmm_net._proj_pose_token_emb.fc_layers.0": {
+      "version": 1
+    },
+    "mmm_net._proj_pose_token_emb.fc_layers.1": {
+      "version": 1
+    },
+    "mmm_net._proj_pose_token_emb.relu_layers": {
+      "version": 1
+    },
+    "mmm_net._proj_pose_token_emb.relu_layers.0": {
+      "version": 1
+    },
+    "mmm_net._proj_pose_token_emb.relu_layers.1": {
+      "version": 1
+    },
+    "mmm_net._proj_local_root_values": {
+      "version": 1
+    },
+    "mmm_net._proj_local_pose": {
+      "version": 1
+    },
+    "mmm_net._proj_num_valid_positions": {
+      "version": 1
+    },
+    "mmm_net._proj_input": {
+      "version": 1
+    },
+    "mmm_net._proj_input.0": {
+      "version": 1
+    },
+    "mmm_net._proj_input.1": {
+      "version": 1
+    },
+    "mmm_net._position_emb": {
+      "version": 1
+    },
+    "mmm_net._position_emb.dropout": {
+      "version": 1
+    },
+    "mmm_net._proj_pose_output_logit": {
+      "version": 1
+    },
+    "motion_rep": {
+      "version": 1
+    },
+    "motion_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.stats": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.global_motion_rep": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.global_motion_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.global_motion_rep.stats": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.local_motion_rep": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.local_motion_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.local_motion_rep.stats": {
+      "version": 1
+    },
+    "motion_rep.stats": {
+      "version": 1
+    },
+    "global_motion_rep": {
+      "version": 1
+    },
+    "global_motion_rep.skeleton": {
+      "version": 1
+    },
+    "global_motion_rep.stats": {
+      "version": 1
+    },
+    "local_motion_rep": {
+      "version": 1
+    },
+    "local_motion_rep.skeleton": {
+      "version": 1
+    },
+    "local_motion_rep.stats": {
+      "version": 1
+    }
+  },
+  "omitted_training_state": [
+    "loops",
+    "callbacks",
+    "optimizer_states",
+    "lr_schedulers"
+  ]
+}

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8537d9d33a880ff1df6c16c28ed061d60c461a5918d70b1cf99251ff91a10a10
+size 546109841

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml

@@ -0,0 +1,250 @@
+model:
+  mmm_network:
+    _target_: rnms.mmm.neural_modules.mmm_net_pose_v6.pose_mmm_network
+    motion_rep: ???
+    args:
+      pose_root_mode: ${model.args.pose_root_mode}
+      min_tokens: ${model.args.min_tokens}
+      max_tokens: ${model.args.max_tokens}
+      down_t: ${model.args.down_t}
+      pose_token_mlp_num_layers: 2
+      root_feat_width: 256
+      pose_feat_width: 640
+      token_length_feat_width: 128
+      text_emb_dim: 4096
+      text_embeddings: ${data.text_embeddings}
+      local_pose_feature: ${model.args.local_pose_feature}
+      cond_root_feature: ${model.args.cond_root_feature}
+      cond_root_feature_is_from_motion_rep: ${model.args.cond_root_feature_is_from_motion_rep}
+      n_embd: 1024
+      n_head: 16
+      n_layers: 16
+      pose_vqvae:
+        nb_code: ${model.pose_vqvae_network.nb_code}
+        code_dim: ${model.pose_vqvae_network.code_dim}
+        num_heads: ${model.pose_vqvae_network.num_heads}
+        has_codebook: ${oc.select:model.pose_vqvae_network.has_codebook,true}
+      root_vqvae:
+        nb_code: 1024
+        code_dim: 60
+        num_heads: 1
+  pose_vqvae_network:
+    _target_: rnms.vqvae.neural_modules.bones_vqvae_v6.VQVAE
+    pose_root_mode: pose
+    motion_rep: ???
+    encoder_state_dim: 241
+    decoder_state_dim: 329
+    decoder_target_cond_dim: 241
+    decoder_external_cond_dim: 2
+    feature_mode:
+    - joint_positions_and_rotations_and_hip_height
+    - pose
+    - joint_positions_and_rotations_and_hip_height
+    - root_without_hip_height_without_heading
+    quantizer_strategy: multihead_ema_reset
+    quantizer_mu: 0.99
+    nb_code: 1048576
+    code_dim: 160
+    output_emb_width: 160
+    down_t: 2
+    stride_t: 2
+    width: 512
+    depth: 4
+    dilation_growth_rate: 3
+    activation: relu
+    num_heads: 5
+    kmeans_init: false
+    norm: None
+    calculate_per_head_perplexity: true
+    cond_fusion_last_layer: false
+  _target_: rnms.mmm.models.MMM_pose_v6.MaskedMotionModel
+  motion_rep: ???
+  root_vqvae_network: null
+  pose_vqvae_motion_rep: local
+  denoiser:
+    backbone:
+      llm_shape: null
+  args:
+    min_tokens: 6
+    max_tokens: 16
+    down_t: 2
+    vqvae_model_ckpt_path: /lustre/fsw/portfolios/edgeai/projects/edgeai_a2x/users/odionne/rnms/results/vqvae-v6-soma-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.ckpt
+    pose_vqvae_motion_rep: ${model.pose_vqvae_motion_rep}
+    incorrect_token_ratio_min: 0.0
+    incorrect_token_ratio_max: 1.0
+    masked_token_ratio: 0.8
+    batchsize_mul_factor: 2
+    pose_root_mode: pose
+    local_pose_feature: joint_positions_and_rotations_and_hip_height
+    cond_root_feature: root_without_hip_height
+    cond_root_feature_is_from_motion_rep: global
+    max_num_start_keyframes: 4
+    no_start_keyframe_prob: 0.0
+    max_num_end_keyframes: 4
+    no_end_keyframe_prob: 0.0
+    max_num_middle_keyframes: 8
+    no_middle_keyframe_prob: 0.5
+    prob_provide_text_emb: 0.2
+    keyframe_num_warmup_steps: 200000
+    floor_estimation: null
+    floor_estimation_window_time: 1.0
+  optimizer:
+    _target_: adam_atan2_pytorch.AdamAtan2
+    _partial_: true
+    lr: 2.0e-05
+    weight_decay: 0.0
+  scheduler:
+    _target_: mfm.train.scheduler.WarmupCosineScheduler
+    _partial_: true
+    num_warmup_steps: 10000
+    num_training_steps: ${trainer.max_steps}
+    final_lr: 1.0e-07
+    last_epoch: -1
+    lt_kwargs:
+      interval: step
+      frequency: 1
+trainer:
+  _target_: pytorch_lightning.Trainer
+  precision: 32
+  enable_progress_bar: false
+  profiler: null
+  check_val_every_n_epoch: null
+  detect_anomaly: false
+  num_sanity_val_steps: 0
+  max_steps: 1500001
+  gradient_clip_val: 0.5
+  log_every_n_steps: 100
+  val_check_interval: 50000
+  devices: 8
+  num_nodes: 4
+  accelerator: gpu
+  strategy: ddp
+  use_distributed_sampler: true
+loggers:
+  out_dir: ${out_dir}
+  cfg_id: ${id}
+  max_steps: ${trainer.max_steps}
+  log_nsteps: ${trainer.log_every_n_steps}
+  use_wandb: false
+  wandb_run: ${wandb_run}
+  wandb_group: null
+  wandb_project: rnms_mmm
+  wandb_entity: nv-gear
+  onelogger_project: mfm
+  wandb_run_name: null
+callbacks:
+  callback_dict:
+    checkpoint_epoch_cb:
+      _target_: mfm.train.callbacks.ckpt.ReadPermissionModelCheckpoint
+      monitor: null
+      dirpath: /lustre/fsw/portfolios/edgeai/projects/edgeai_a2x/users/odionne/rnms/results/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints
+      filename: model-{step:07d}
+      save_last: false
+      save_top_k: -1
+      mode: min
+      every_n_train_steps: 50000
+    learning_rate_monitor:
+      _target_: pytorch_lightning.callbacks.LearningRateMonitor
+      logging_interval: step
+    autoresume:
+      _target_: mfm.train.callbacks.autoresume.AutoResumeCallback
+      autoresume_after: 12600
+      save_checkpoint_on_exception: true
+    mmm_evaluate:
+      _target_: rnms.mmm.callbacks.evaluation.MMM_Evaluate
+      every_n_steps: 50001
+      train_dataloader: ???
+      val_dataloader: ???
+      seed: 2333
+      batch_size: 32
+      max_num_data: 40000
+      max_num_visualization: 32
+      out_dir: ${run_dir}/vis
+    train_end:
+      _target_: mfm.train.callbacks.train_end.TrainEndCallback
+  only_rank_zero:
+  - learning_rate_monitor
+  - mmm_evaluate
+  - train_end
+  need_version: []
+  need_dataloaders:
+  - mmm_evaluate
+  need_ema: []
+  need_mfm_test: []
+  need_log:
+  - mmm_evaluate
+  need_autoresume:
+  - mmm_evaluate
+  need_mask_cond_sampler: []
+  need_one_logger_callback:
+  - autoresume
+dataloader:
+  _target_: torch.utils.data.DataLoader
+  batch_size: 128
+  num_workers: 15
+  shuffle: ???
+  pin_memory: true
+  persistent_workers: true
+  collate_fn:
+    _target_: mfm.data.utils.collate_batch
+    _partial_: true
+    llm_shape: ${model.denoiser.backbone.llm_shape}
+    num_frames: null
+train_split: train_nova
+val_split: unique_test_nova
+data:
+  _target_: mfm.data.motion_dataset.MotionDataset
+  name: rigplay1-v1
+  folder: ${data_root}/${.name}
+  data_variation_folder: anims_uniform
+  text_embeddings: null
+  split: ???
+  motion_sampler:
+    _target_: mfm.data.motion_sampler.MaxDurationRandomCrop
+    max_seconds: 15
+  motion_loading_mode: memmap
+  text_fields:
+    natural_desc_1: 0.125
+    natural_desc_2: 0.125
+    natural_desc_3: 0.125
+    short_description: 0.125
+    nv_overview_desc: 0.5
+  drop_text_prob: 0.1
+  augment_text: false
+  aug_text_suffix: _qwen
+  aug_text_prob: null
+  load_neutral_joints: false
+  timelines_mode: false
+  timelines_name: null
+  dataset_sampler: null
+  dataset_sampler_only_at_train: true
+  to_canonicalize: false
+  randomize_first_heading: true
+  loading_mode: motion_only
+skeleton:
+  base_name: novaskel30
+  folder: ${data.folder}/features/motion_rep/${motion_rep.name}_fps_${fps}/skeleton
+  t_pose: standard
+  orig_prefix: ''
+  name: ${skeleton.orig_prefix}${skeleton.base_name}
+  _target_: mfm.core.skeletons.nova.NovaSkeleton30
+motion_rep:
+  name: ${skeleton.name}_dual_root_global_joints
+  stats:
+    _target_: mfm.core.utils.stats.Stats
+    folder: ${data.folder}/features/motion_rep/${motion_rep.name}_fps_${fps}/stats
+  _target_: mfm.core.motion_reps.dual_root_global_joints.GlobalRootGlobalJoints
+fps: 30
+wandb_run: null
+resume: false
+version: null
+cp: last_epoch
+matmul_precision: high
+seed: 7
+id: ${hydra:runtime.choices.exp}_gpu${trainer.devices}_node${trainer.num_nodes}
+out_dir: /lustre/fsw/portfolios/edgeai/projects/edgeai_a2x/users/odionne/rnms/results/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1
+run_dir: /lustre/fsw/portfolios/edgeai/projects/edgeai_a2x/users/odionne/rnms/results/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0
+pure_testing: false
+config_version: 4
+data_root: /lustre/fsw/portfolios/nvr/projects/nvr_torontoai_humanmotionfm/datasets
+tmr_checkpoints_root: /lustre/fsw/portfolios/nvr/projects/nvr_torontoai_humanmotionfm/models/TMR

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/joints.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57add41fb635ad9b43e85478954c05087b07d57184676ebb5cfe3c5864b5daac
+size 848

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/skeleton/skeleton.yaml

@@ -0,0 +1,6 @@
+base_name: novaskel30
+folder: ${data.folder}/features/motion_rep/${motion_rep.name}_fps_${fps}/skeleton
+t_pose: standard
+orig_prefix: ''
+name: ${skeleton.orig_prefix}${skeleton.base_name}
+_target_: mfm.core.skeletons.nova.NovaSkeleton30

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/mean.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:802f79631ea88a1a920cf44725f5d0e9973a0c8d7adb5359b01bd7a5af57492d
+size 3088

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-pose-nova-4n8g-ea43dd8-fp32-v1/version_0/stats/motion/std.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a97818fea2a10e562de059013b081ecf7943d6815a36396f0d94f54ad26acb6
+size 3088

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.json

@@ -0,0 +1,705 @@
+{
+  "schema_version": 1,
+  "source_checkpoint_sha256": "5059d7949376fe45a95f9b6869e386a7038840f6cfa6d778747c44ae82f87a93",
+  "epoch": 1224,
+  "global_step": 1500000,
+  "pytorch_lightning_version": "2.4.0",
+  "state_dict_metadata": {
+    "": {
+      "version": 1
+    },
+    "mmm_net": {
+      "version": 1
+    },
+    "mmm_net.motion_rep": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.stats": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.global_motion_rep": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.global_motion_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.global_motion_rep.stats": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.local_motion_rep": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.local_motion_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.dual_rep.local_motion_rep.stats": {
+      "version": 1
+    },
+    "mmm_net.motion_rep.stats": {
+      "version": 1
+    },
+    "mmm_net.global_motion_rep": {
+      "version": 1
+    },
+    "mmm_net.global_motion_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.global_motion_rep.stats": {
+      "version": 1
+    },
+    "mmm_net.local_motion_rep": {
+      "version": 1
+    },
+    "mmm_net.local_motion_rep.skeleton": {
+      "version": 1
+    },
+    "mmm_net.local_motion_rep.stats": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.0": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.0.self_attn": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.0.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.0.linear1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.0.dropout": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.0.linear2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.0.norm1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.0.norm2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.0.dropout1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.0.dropout2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.1.self_attn": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.1.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.1.linear1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.1.dropout": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.1.linear2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.1.norm1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.1.norm2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.1.dropout1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.1.dropout2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.2.self_attn": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.2.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.2.linear1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.2.dropout": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.2.linear2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.2.norm1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.2.norm2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.2.dropout1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.2.dropout2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.3": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.3.self_attn": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.3.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.3.linear1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.3.dropout": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.3.linear2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.3.norm1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.3.norm2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.3.dropout1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.3.dropout2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.4": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.4.self_attn": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.4.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.4.linear1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.4.dropout": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.4.linear2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.4.norm1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.4.norm2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.4.dropout1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.4.dropout2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.5": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.5.self_attn": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.5.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.5.linear1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.5.dropout": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.5.linear2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.5.norm1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.5.norm2": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.5.dropout1": {
+      "version": 1
+    },
+    "mmm_net._shared_transformer_model.layers.5.dropout2": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.0": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.0.self_attn": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.0.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.0.linear1": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.0.dropout": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.0.linear2": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.0.norm1": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.0.norm2": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.0.dropout1": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.0.dropout2": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.1": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.1.self_attn": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.1.self_attn.out_proj": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.1.linear1": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.1.dropout": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.1.linear2": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.1.norm1": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.1.norm2": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.1.dropout1": {
+      "version": 1
+    },
+    "mmm_net._root_token_transformer_model.layers.1.dropout2": {
+      "version": 1
+    },
+    "mmm_net._conv_output": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.0": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.0": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.0.norm1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.0.norm2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.0.activation1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.0.activation2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.0.conv1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.0.conv2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.1.norm1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.1.norm2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.1.activation1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.1.activation2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.1.conv1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.1.conv2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.2.norm1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.2.norm2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.2.activation1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.2.activation2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.2.conv1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.2.conv2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.3": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.3.norm1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.3.norm2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.3.activation1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.3.activation2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.3.conv1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.0.model.3.conv2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.2.2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.0": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.0.norm1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.0.norm2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.0.activation1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.0.activation2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.0.conv1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.0.conv2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.1.norm1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.1.norm2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.1.activation1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.1.activation2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.1.conv1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.1.conv2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.2.norm1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.2.norm2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.2.activation1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.2.activation2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.2.conv1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.2.conv2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.3": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.3.norm1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.3.norm2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.3.activation1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.3.activation2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.3.conv1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.0.model.3.conv2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.3.2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.4": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.5": {
+      "version": 1
+    },
+    "mmm_net._conv_output.model.6": {
+      "version": 1
+    },
+    "mmm_net._conv_output.external_cond_blocks": {
+      "version": 1
+    },
+    "mmm_net._conv_output.external_cond_blocks.0": {
+      "version": 1
+    },
+    "mmm_net._conv_output.external_cond_blocks.1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.external_cond_blocks.2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.external_cond_blocks.3": {
+      "version": 1
+    },
+    "mmm_net._conv_output.target_cond_blocks": {
+      "version": 1
+    },
+    "mmm_net._conv_output.target_cond_blocks.0": {
+      "version": 1
+    },
+    "mmm_net._conv_output.target_cond_blocks.1": {
+      "version": 1
+    },
+    "mmm_net._conv_output.target_cond_blocks.2": {
+      "version": 1
+    },
+    "mmm_net._conv_output.target_cond_blocks.3": {
+      "version": 1
+    },
+    "mmm_net._proj_local_pose": {
+      "version": 1
+    },
+    "mmm_net._proj_local_root_value": {
+      "version": 1
+    },
+    "mmm_net._proj_global_root_value": {
+      "version": 1
+    },
+    "mmm_net._proj_start_input": {
+      "version": 1
+    },
+    "mmm_net._proj_start_input.forward_projection": {
+      "version": 1
+    },
+    "mmm_net._proj_start_input.fc_layers": {
+      "version": 1
+    },
+    "mmm_net._proj_start_input.fc_layers.0": {
+      "version": 1
+    },
+    "mmm_net._proj_start_input.fc_layers.1": {
+      "version": 1
+    },
+    "mmm_net._proj_start_input.relu_layers": {
+      "version": 1
+    },
+    "mmm_net._proj_start_input.relu_layers.0": {
+      "version": 1
+    },
+    "mmm_net._proj_start_input.relu_layers.1": {
+      "version": 1
+    },
+    "mmm_net._proj_end_input": {
+      "version": 1
+    },
+    "mmm_net._proj_end_input.forward_projection": {
+      "version": 1
+    },
+    "mmm_net._proj_end_input.fc_layers": {
+      "version": 1
+    },
+    "mmm_net._proj_end_input.fc_layers.0": {
+      "version": 1
+    },
+    "mmm_net._proj_end_input.fc_layers.1": {
+      "version": 1
+    },
+    "mmm_net._proj_end_input.relu_layers": {
+      "version": 1
+    },
+    "mmm_net._proj_end_input.relu_layers.0": {
+      "version": 1
+    },
+    "mmm_net._proj_end_input.relu_layers.1": {
+      "version": 1
+    },
+    "mmm_net._input_position_emb": {
+      "version": 1
+    },
+    "mmm_net._proj_input_num_tokens": {
+      "version": 1
+    },
+    "mmm_net._position_emb": {
+      "version": 1
+    },
+    "mmm_net._position_emb.dropout": {
+      "version": 1
+    },
+    "mmm_net._proj_num_token_output_logit": {
+      "version": 1
+    },
+    "mmm_net._middle_token_emb": {
+      "version": 1
+    },
+    "motion_rep": {
+      "version": 1
+    },
+    "motion_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.stats": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.global_motion_rep": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.global_motion_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.global_motion_rep.stats": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.local_motion_rep": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.local_motion_rep.skeleton": {
+      "version": 1
+    },
+    "motion_rep.dual_rep.local_motion_rep.stats": {
+      "version": 1
+    },
+    "motion_rep.stats": {
+      "version": 1
+    },
+    "global_motion_rep": {
+      "version": 1
+    },
+    "global_motion_rep.skeleton": {
+      "version": 1
+    },
+    "global_motion_rep.stats": {
+      "version": 1
+    },
+    "local_motion_rep": {
+      "version": 1
+    },
+    "local_motion_rep.skeleton": {
+      "version": 1
+    },
+    "local_motion_rep.stats": {
+      "version": 1
+    }
+  },
+  "omitted_training_state": [
+    "loops",
+    "callbacks",
+    "optimizer_states",
+    "lr_schedulers"
+  ]
+}

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints/model-step=1500000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a48d48c3d8ff98c825ca315797644789ae37f2db9069779a3118f42e84f4f29e
+size 161695084

soma/soma-v6-nova-4n8g-ea43dd8-fp32-v1/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/config.yaml

@@ -0,0 +1,228 @@
+model:
+  mmm_network:
+    _target_: rnms.mmm.neural_modules.mmm_net_root_v6.root_mmm_network
+    motion_rep: ???
+    args:
+      min_tokens: ${model.args.min_tokens}
+      max_tokens: ${model.args.max_tokens}
+      down_t: ${model.args.down_t}
+      use_hard_num_token_emb_for_root_prediction: ${model.args.use_hard_num_token_emb_for_root_prediction}
+      local_root_feature: ${model.args.local_root_feature}
+      global_root_feature: ${model.args.global_root_feature}
+      local_pose_feature: ${model.args.local_pose_feature}
+      pose_feat_dim: 256
+      local_root_feat_dim: 64
+      global_root_feat_dim: 64
+      text_emb_dim: 4096
+      text_embeddings: ${data.text_embeddings}
+      input_feat_mlp_num_layers: 2
+      n_embd: 512
+      n_head: 16
+      n_layers_shared: 6
+      n_layers_root_token: 2
+      width: 512
+      depth: 4
+      dilation_growth_rate: 3
+      activation: relu
+      norm: None
+      pose_vqvae:
+        nb_code: 1024
+        code_dim: 60
+        num_heads: 1
+      root_vqvae:
+        nb_code: 1024
+        code_dim: 60
+        num_heads: 1
+  _target_: rnms.mmm.models.MMM_root_v6.MaskedMotionModel
+  motion_rep: ???
+  root_vqvae_network: null
+  pose_vqvae_network: null
+  denoiser:
+    backbone:
+      llm_shape: null
+  args:
+    pose_root_mode: root
+    local_pose_feature: joint_positions_and_rotations_and_hip_height
+    local_root_feature: root
+    global_root_feature: root
+    min_tokens: 6
+    max_tokens: 16
+    use_hard_num_token_emb_for_root_prediction: true
+    min_off_target_tokens_to_sample: 25
+    max_off_target_tokens_to_sample: 40
+    prob_off_range_target_token: 0.0
+    prob_provide_num_tokens: 0.0
+    down_t: 2
+    vqvae_model_ckpt_path: null
+    batchsize_mul_factor: 2
+    max_num_start_keyframes: 4
+    no_start_keyframe_prob: 0.0
+    max_num_end_keyframes: 4
+    no_end_keyframe_prob: 0.0
+    prob_provide_text_emb: 0.0
+    num_token_loss_coeff: 1.0
+    global_root_loss_coeff: 2.0
+    local_root_loss_coeff: 1.0
+    keyframe_num_warmup_steps: 200000
+    floor_estimation: null
+    floor_estimation_window_time: 1.0
+  optimizer:
+    _target_: adam_atan2_pytorch.AdamAtan2
+    _partial_: true
+    lr: 2.0e-05
+    weight_decay: 0.0
+  scheduler:
+    _target_: mfm.train.scheduler.WarmupCosineScheduler
+    _partial_: true
+    num_warmup_steps: 10000
+    num_training_steps: ${trainer.max_steps}
+    final_lr: 1.0e-07
+    last_epoch: -1
+    lt_kwargs:
+      interval: step
+      frequency: 1
+trainer:
+  _target_: pytorch_lightning.Trainer
+  precision: 32
+  enable_progress_bar: false
+  profiler: null
+  check_val_every_n_epoch: null
+  detect_anomaly: false
+  num_sanity_val_steps: 0
+  max_steps: 1500001
+  gradient_clip_val: 0.5
+  log_every_n_steps: 100
+  val_check_interval: 50000
+  devices: 8
+  num_nodes: 4
+  accelerator: gpu
+  strategy: ddp
+  use_distributed_sampler: true
+loggers:
+  out_dir: ${out_dir}
+  cfg_id: ${id}
+  max_steps: ${trainer.max_steps}
+  log_nsteps: ${trainer.log_every_n_steps}
+  use_wandb: false
+  wandb_run: ${wandb_run}
+  wandb_group: null
+  wandb_project: rnms_mmm
+  wandb_entity: nv-gear
+  onelogger_project: mfm
+  wandb_run_name: null
+callbacks:
+  callback_dict:
+    checkpoint_epoch_cb:
+      _target_: mfm.train.callbacks.ckpt.ReadPermissionModelCheckpoint
+      monitor: null
+      dirpath: /lustre/fsw/portfolios/edgeai/projects/edgeai_a2x/users/odionne/rnms/results/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0/checkpoints
+      filename: model-{step:07d}
+      save_last: false
+      save_top_k: -1
+      mode: min
+      every_n_train_steps: 50000
+    learning_rate_monitor:
+      _target_: pytorch_lightning.callbacks.LearningRateMonitor
+      logging_interval: step
+    autoresume:
+      _target_: mfm.train.callbacks.autoresume.AutoResumeCallback
+      autoresume_after: 12600
+      save_checkpoint_on_exception: true
+    mmm_evaluate:
+      _target_: rnms.mmm.callbacks.evaluation.MMM_Evaluate
+      every_n_steps: 50001
+      train_dataloader: ???
+      val_dataloader: ???
+      seed: 2333
+      batch_size: 32
+      max_num_data: 40000
+      max_num_visualization: 32
+      out_dir: ${run_dir}/vis
+    train_end:
+      _target_: mfm.train.callbacks.train_end.TrainEndCallback
+  only_rank_zero:
+  - learning_rate_monitor
+  - mmm_evaluate
+  - train_end
+  need_version: []
+  need_dataloaders:
+  - mmm_evaluate
+  need_ema: []
+  need_mfm_test: []
+  need_log:
+  - mmm_evaluate
+  need_autoresume:
+  - mmm_evaluate
+  need_mask_cond_sampler: []
+  need_one_logger_callback:
+  - autoresume
+dataloader:
+  _target_: torch.utils.data.DataLoader
+  batch_size: 8
+  num_workers: 15
+  shuffle: ???
+  pin_memory: true
+  persistent_workers: true
+  collate_fn:
+    _target_: mfm.data.utils.collate_batch
+    _partial_: true
+    llm_shape: ${model.denoiser.backbone.llm_shape}
+    num_frames: null
+train_split: train_nova
+val_split: unique_test_nova
+data:
+  _target_: mfm.data.motion_dataset.MotionDataset
+  name: rigplay1-v1
+  folder: ${data_root}/${.name}
+  data_variation_folder: anims_uniform
+  text_embeddings: null
+  split: ???
+  motion_sampler:
+    _target_: mfm.data.motion_sampler.MaxDurationRandomCrop
+    max_seconds: 15
+  motion_loading_mode: memmap
+  text_fields:
+    natural_desc_1: 0.125
+    natural_desc_2: 0.125
+    natural_desc_3: 0.125
+    short_description: 0.125
+    nv_overview_desc: 0.5
+  drop_text_prob: 0.1
+  augment_text: false
+  aug_text_suffix: _qwen
+  aug_text_prob: null
+  load_neutral_joints: false
+  timelines_mode: false
+  timelines_name: null
+  dataset_sampler: null
+  dataset_sampler_only_at_train: true
+  to_canonicalize: false
+  randomize_first_heading: true
+  loading_mode: motion_only
+skeleton:
+  base_name: novaskel30
+  folder: ${data.folder}/features/motion_rep/${motion_rep.name}_fps_${fps}/skeleton
+  t_pose: standard
+  orig_prefix: ''
+  name: ${skeleton.orig_prefix}${skeleton.base_name}
+  _target_: mfm.core.skeletons.nova.NovaSkeleton30
+motion_rep:
+  name: ${skeleton.name}_dual_root_global_joints
+  stats:
+    _target_: mfm.core.utils.stats.Stats
+    folder: ${data.folder}/features/motion_rep/${motion_rep.name}_fps_${fps}/stats
+  _target_: mfm.core.motion_reps.dual_root_global_joints.GlobalRootGlobalJoints
+fps: 30
+wandb_run: null
+resume: false
+version: null
+cp: last_epoch
+matmul_precision: high
+seed: 7
+id: ${hydra:runtime.choices.exp}_gpu${trainer.devices}_node${trainer.num_nodes}
+out_dir: /lustre/fsw/portfolios/edgeai/projects/edgeai_a2x/users/odionne/rnms/results/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1
+run_dir: /lustre/fsw/portfolios/edgeai/projects/edgeai_a2x/users/odionne/rnms/results/mmm-v6-soma-root-nova-4n8g-ea43dd8-fp32-v1/version_0
+pure_testing: false
+config_version: 4
+data_root: /lustre/fsw/portfolios/nvr/projects/nvr_torontoai_humanmotionfm/datasets
+tmr_checkpoints_root: /lustre/fsw/portfolios/nvr/projects/nvr_torontoai_humanmotionfm/models/TMR