Add private v-dataset inference checkpoint pair and matching configuration
Browse files- README.md +11 -0
- v-dataset/LICENSE +29 -0
- v-dataset/README.md +15 -0
- v-dataset/SHA256SUMS +5 -0
- v-dataset/configuration.json +252 -0
- v-dataset/init.ckpt +3 -0
- v-dataset/refine_solver.ckpt +3 -0
README.md
CHANGED
|
@@ -34,6 +34,17 @@ The full pipeline uses first-frame Init, temporal Refine in 16-frame windows and
|
|
| 34 |
|
| 35 |
The checkpoints contain inference network parameters and buffers. SMPL-H/MANO models and DPoser assets must be prepared separately; they are not included in these checkpoint files.
|
| 36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
## Usage
|
| 38 |
|
| 39 |
Prepare the environment, body model and checkpoints with [Install.md](https://github.com/WangLongZJU/DirtyMoCap/blob/main/Install.md). Run this command from the DirtyMoCap code repository root:
|
|
|
|
| 34 |
|
| 35 |
The checkpoints contain inference network parameters and buffers. SMPL-H/MANO models and DPoser assets must be prepared separately; they are not included in these checkpoint files.
|
| 36 |
|
| 37 |
+
## v-dataset checkpoints
|
| 38 |
+
|
| 39 |
+
The `v-dataset` pair and matching `configs/inference_dataset.yaml` provide the pipeline used to generate HKMALA-Motion. It uses first-frame Init, temporal Refine and the full Solver; `betas_optimize_windows` is 10. The `v1` release remains available with its own configuration.
|
| 40 |
+
|
| 41 |
+
```bash
|
| 42 |
+
python tools/fetch_checkpoints.py --version v-dataset
|
| 43 |
+
python -m src.infer --config configs/inference_dataset.yaml --input my_markers.npz --output outputs/motion.npz --gpu 0
|
| 44 |
+
```
|
| 45 |
+
|
| 46 |
+
See the [version description](https://huggingface.co/aLongZJU/DirtyMoCap/blob/main/v-dataset/README.md) and [inference guide](https://github.com/WangLongZJU/DirtyMoCap/blob/main/Inference.md#hkmala-motion-inference-with-v-dataset). Model weights use the pretrained checkpoint license. HKMALA-Motion numerical recordings have their own dataset license and access requirements.
|
| 47 |
+
|
| 48 |
## Usage
|
| 49 |
|
| 50 |
Prepare the environment, body model and checkpoints with [Install.md](https://github.com/WangLongZJU/DirtyMoCap/blob/main/Install.md). Run this command from the DirtyMoCap code repository root:
|
v-dataset/LICENSE
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DirtyMoCap Pretrained Model License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2026 The DirtyMoCap Authors.
|
| 4 |
+
|
| 5 |
+
## Scope
|
| 6 |
+
|
| 7 |
+
These terms apply to pretrained DirtyMoCap checkpoints distributed with this license, including `v1/init.ckpt` and `v1/refine_solver.ckpt`. Source code and third-party resources are licensed separately.
|
| 8 |
+
|
| 9 |
+
## Permitted purposes
|
| 10 |
+
|
| 11 |
+
The provided checkpoints may be used only for **non-commercial scientific research, non-commercial education or non-commercial artistic projects**. This restriction also applies to use of modified or fine-tuned versions of these checkpoints. Commercial use is not authorized by these terms.
|
| 12 |
+
|
| 13 |
+
## Third-party resources
|
| 14 |
+
|
| 15 |
+
Training used AMASS and GRAB motion data together with SMPL-H/MANO. Users must comply with the applicable upstream terms:
|
| 16 |
+
|
| 17 |
+
- [AMASS license](https://amass.is.tue.mpg.de/license.html)
|
| 18 |
+
- [GRAB license](https://grab.is.tue.mpg.de/license.html)
|
| 19 |
+
- [SMPL-H/MANO license](https://mano.is.tue.mpg.de/license.html)
|
| 20 |
+
|
| 21 |
+
The DirtyMoCap authors grant no rights in third-party datasets, body models, software or pretrained assets. Obtain those resources from their official providers under their applicable licenses. This document does not authorize redistribution of third-party resources or replace any permission required from their rights holders.
|
| 22 |
+
|
| 23 |
+
## Separate source-code license
|
| 24 |
+
|
| 25 |
+
The Apache License 2.0 for the original DirtyMoCap source code does not apply to these checkpoints. Third-party components retain their own terms.
|
| 26 |
+
|
| 27 |
+
## Disclaimer
|
| 28 |
+
|
| 29 |
+
The checkpoints are provided as is, without warranties of any kind. To the extent permitted by applicable law, the DirtyMoCap authors are not liable for claims or damages arising from their use.
|
v-dataset/README.md
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DirtyMoCap v-dataset
|
| 2 |
+
|
| 3 |
+
Paired Init and Refine/Solver inference weights for the pipeline used to generate HKMALA-Motion. Obtain the environment and third-party assets separately with the code repository's Install.md.
|
| 4 |
+
|
| 5 |
+
```bash
|
| 6 |
+
python tools/fetch_checkpoints.py --version v-dataset
|
| 7 |
+
python tools/check.py --version v-dataset
|
| 8 |
+
python -m src.infer --config configs/inference_dataset.yaml --input my_markers.npz --output outputs/motion.npz --gpu 0
|
| 9 |
+
```
|
| 10 |
+
|
| 11 |
+
The profile uses first-frame Init, stateful Refine in overlapping 16-frame windows, and Solver in 32-frame windows with a 4-frame overlap and 10 steps. Shape optimization uses `betas_optimize_windows=10`. `configuration.json` records the paired code configuration. Seeds and CUDA operators can cause numerical differences between repeated inferences.
|
| 12 |
+
|
| 13 |
+
The files contain neural parameters and buffers, with fingerprints for separately reconstructed body-model assets. Body-model tensors, training recovery state, SMPL-H/MANO models and DPoser assets are not included. The Init checkpoint is shared with v1, and both version folders are independently usable with their matching code configurations.
|
| 14 |
+
|
| 15 |
+
The [DirtyMoCap Pretrained Model License](https://huggingface.co/aLongZJU/DirtyMoCap/blob/main/v-dataset/LICENSE) applies to these checkpoints. HKMALA-Motion recordings have a separate dataset license. This version does not change the default v1 release.
|
v-dataset/SHA256SUMS
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
e7746e2efb306e52b849dd1bc026c5a5afc090c6019451457b0e1f8238283a02 init.ckpt
|
| 2 |
+
40a9fc631b131ac1f9b86bb83d72f3e9c37c779d1a63c285ee7eaa00cbf5d838 refine_solver.ckpt
|
| 3 |
+
f807f1018535608d01a4d317348e476d1e4a6e28aa9de59cf7e7aaf6d4f73c6a LICENSE
|
| 4 |
+
c6f9daab08f596acd80438c60492a5e6ee93f3e3a6f70c4504e765fdfea5b7cd README.md
|
| 5 |
+
6d3a7d841f15866d65d407a74f9b711c0dcba8c318bffa9ca3ba85c462a04ba6 configuration.json
|
v-dataset/configuration.json
ADDED
|
@@ -0,0 +1,252 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"inference": {
|
| 3 |
+
"checkpoints": {
|
| 4 |
+
"init": "assets/checkpoints/v-dataset/init.ckpt",
|
| 5 |
+
"refine_solver": "assets/checkpoints/v-dataset/refine_solver.ckpt"
|
| 6 |
+
},
|
| 7 |
+
"assets": {
|
| 8 |
+
"body_models": "assets/body_models",
|
| 9 |
+
"prior": "assets/dposer_x"
|
| 10 |
+
},
|
| 11 |
+
"models": {
|
| 12 |
+
"init": "configs/models/init.yaml",
|
| 13 |
+
"refine_solver": "configs/models/refine_solver_dataset.yaml",
|
| 14 |
+
"anchors": "configs/anchors.json"
|
| 15 |
+
},
|
| 16 |
+
"runtime": {
|
| 17 |
+
"gpu": 0,
|
| 18 |
+
"memory_budget_gib": 6.0,
|
| 19 |
+
"minimum_free_gib": 8.0,
|
| 20 |
+
"context_allowance_gib": 2.0,
|
| 21 |
+
"offload_modules": true,
|
| 22 |
+
"cuda_home": "/usr/local/cuda-12.8",
|
| 23 |
+
"cuda_build_dir": "workspace/cuda_build"
|
| 24 |
+
},
|
| 25 |
+
"inference": {
|
| 26 |
+
"mode": "full",
|
| 27 |
+
"init_mode": "first_frame",
|
| 28 |
+
"seed": 0,
|
| 29 |
+
"save_mesh": true
|
| 30 |
+
}
|
| 31 |
+
},
|
| 32 |
+
"refine_solver": {
|
| 33 |
+
"model": {
|
| 34 |
+
"checkpoint_source_sha256": "95749236179f66805081a6480153e3cf476fbe88e7ffcb9dcf2f29f60cd76490",
|
| 35 |
+
"class_name": "sequence_model.CorrectionModel",
|
| 36 |
+
"smplx_type": "smplh",
|
| 37 |
+
"smplx_path": "assets/body_models",
|
| 38 |
+
"flat_hand": false,
|
| 39 |
+
"backbone": {
|
| 40 |
+
"enc_channels": [
|
| 41 |
+
32,
|
| 42 |
+
64,
|
| 43 |
+
128,
|
| 44 |
+
256,
|
| 45 |
+
512
|
| 46 |
+
],
|
| 47 |
+
"dec_channels": [
|
| 48 |
+
256,
|
| 49 |
+
256,
|
| 50 |
+
256,
|
| 51 |
+
256
|
| 52 |
+
]
|
| 53 |
+
},
|
| 54 |
+
"use_rope": true,
|
| 55 |
+
"ori_emb": false,
|
| 56 |
+
"win_size": 16,
|
| 57 |
+
"use_state": true,
|
| 58 |
+
"state_size": 768,
|
| 59 |
+
"to_cano_with_firstR": false,
|
| 60 |
+
"use_init_model_for_training": true,
|
| 61 |
+
"use_init_model_for_training_prob": 1.0,
|
| 62 |
+
"init_train_config": "configs/models/init.yaml",
|
| 63 |
+
"init_checkpoint_path": "assets/checkpoints/init.ckpt",
|
| 64 |
+
"staged_training": {
|
| 65 |
+
"phase": 3,
|
| 66 |
+
"n_freeze_drop": 0,
|
| 67 |
+
"freeze_refine_backbone": true,
|
| 68 |
+
"freeze_beta_head": false,
|
| 69 |
+
"freeze_optimize": false,
|
| 70 |
+
"optimize_to_joints": {
|
| 71 |
+
"pred_pose": false,
|
| 72 |
+
"activation_checkpoint": false,
|
| 73 |
+
"betas_optimize_windows": 10,
|
| 74 |
+
"win_size": 32,
|
| 75 |
+
"overlap_size": 4,
|
| 76 |
+
"opt_iters": 10,
|
| 77 |
+
"grid_size": 0.005,
|
| 78 |
+
"use_prior": true,
|
| 79 |
+
"use_joint_smooth": true,
|
| 80 |
+
"use_smooth": false,
|
| 81 |
+
"use_learnable_damping_weight": false,
|
| 82 |
+
"damping_weight_min": 0.001,
|
| 83 |
+
"damping_weight_max": 10.0,
|
| 84 |
+
"damping_weight_init": 0.01,
|
| 85 |
+
"use_learnable_identity_weight": false,
|
| 86 |
+
"identity_weight_min": 1e-06,
|
| 87 |
+
"identity_weight_max": 10.0,
|
| 88 |
+
"identity_weight_init": 0.0001,
|
| 89 |
+
"prior_t_min": 0.04,
|
| 90 |
+
"prior_t_max": 0.12,
|
| 91 |
+
"prior_aux_weight": 0.05,
|
| 92 |
+
"smooth_aux_weight": 2.0,
|
| 93 |
+
"smooth_aux_target": "gt_acc",
|
| 94 |
+
"joint_smooth_aux_weight": 5.0,
|
| 95 |
+
"joint_smooth_aux_target": "gt_acc",
|
| 96 |
+
"step_weight_mode": "single_level_linear",
|
| 97 |
+
"step_weight_interval": 3.0,
|
| 98 |
+
"use_hand_points_for_beta_jacobian": false
|
| 99 |
+
}
|
| 100 |
+
},
|
| 101 |
+
"grid_size": [
|
| 102 |
+
0.01,
|
| 103 |
+
0.005,
|
| 104 |
+
0.001
|
| 105 |
+
],
|
| 106 |
+
"num_joints": 113,
|
| 107 |
+
"n_iters": 1,
|
| 108 |
+
"optim_input_noise": {
|
| 109 |
+
"enabled": false,
|
| 110 |
+
"sample_prob": 0.3,
|
| 111 |
+
"prediction_error_noise": {
|
| 112 |
+
"prob": 0.5,
|
| 113 |
+
"min_frames": 0.3,
|
| 114 |
+
"max_frames": 0.6,
|
| 115 |
+
"min_joints": 0.3,
|
| 116 |
+
"max_joints": 0.6,
|
| 117 |
+
"offset_scale": [
|
| 118 |
+
0.002,
|
| 119 |
+
0.005
|
| 120 |
+
],
|
| 121 |
+
"drift_prob": 0.0
|
| 122 |
+
},
|
| 123 |
+
"joint_noise": {
|
| 124 |
+
"prob": 0.4,
|
| 125 |
+
"min_frames": 0.2,
|
| 126 |
+
"max_frames": 0.3,
|
| 127 |
+
"min_joints": 0.05,
|
| 128 |
+
"max_joints": 0.1,
|
| 129 |
+
"offset_scale": [
|
| 130 |
+
0.2,
|
| 131 |
+
0.5
|
| 132 |
+
],
|
| 133 |
+
"drift_prob": 0.5
|
| 134 |
+
},
|
| 135 |
+
"limb_noise": {
|
| 136 |
+
"prob": 0.4,
|
| 137 |
+
"min_frames": 0.2,
|
| 138 |
+
"max_frames": 0.3,
|
| 139 |
+
"min_group": 1,
|
| 140 |
+
"max_group": 2,
|
| 141 |
+
"offset_scale": [
|
| 142 |
+
0.01,
|
| 143 |
+
0.05
|
| 144 |
+
],
|
| 145 |
+
"drift_prob": 0.5
|
| 146 |
+
},
|
| 147 |
+
"global_noise": {
|
| 148 |
+
"prob": 0.4,
|
| 149 |
+
"min_frames": 0.1,
|
| 150 |
+
"max_frames": 0.2,
|
| 151 |
+
"offset_scale": [
|
| 152 |
+
0.02,
|
| 153 |
+
0.05
|
| 154 |
+
],
|
| 155 |
+
"drift_prob": 0.5
|
| 156 |
+
}
|
| 157 |
+
},
|
| 158 |
+
"cross_window_init_noise": {
|
| 159 |
+
"enabled": false,
|
| 160 |
+
"sample_prob": 0.3,
|
| 161 |
+
"joint_noise": {
|
| 162 |
+
"prob": 0.4,
|
| 163 |
+
"min_frames": 0.2,
|
| 164 |
+
"max_frames": 0.3,
|
| 165 |
+
"min_joints": 0.05,
|
| 166 |
+
"max_joints": 0.1,
|
| 167 |
+
"offset_scale": [
|
| 168 |
+
0.2,
|
| 169 |
+
0.5
|
| 170 |
+
],
|
| 171 |
+
"drift_prob": 0.5
|
| 172 |
+
},
|
| 173 |
+
"limb_noise": {
|
| 174 |
+
"prob": 0.4,
|
| 175 |
+
"min_frames": 0.2,
|
| 176 |
+
"max_frames": 0.3,
|
| 177 |
+
"min_group": 1,
|
| 178 |
+
"max_group": 2,
|
| 179 |
+
"offset_scale": [
|
| 180 |
+
0.01,
|
| 181 |
+
0.05
|
| 182 |
+
],
|
| 183 |
+
"drift_prob": 0.5
|
| 184 |
+
},
|
| 185 |
+
"global_noise": {
|
| 186 |
+
"prob": 0.4,
|
| 187 |
+
"min_frames": 0.1,
|
| 188 |
+
"max_frames": 0.2,
|
| 189 |
+
"offset_scale": [
|
| 190 |
+
0.02,
|
| 191 |
+
0.05
|
| 192 |
+
],
|
| 193 |
+
"drift_prob": 0.5
|
| 194 |
+
}
|
| 195 |
+
},
|
| 196 |
+
"state_feat_noise": {
|
| 197 |
+
"enabled": false,
|
| 198 |
+
"sample_prob": 0.15,
|
| 199 |
+
"reset_prob": 0.3,
|
| 200 |
+
"decay_prob": 0.7,
|
| 201 |
+
"decay_scale": [
|
| 202 |
+
0.5,
|
| 203 |
+
1.0
|
| 204 |
+
]
|
| 205 |
+
},
|
| 206 |
+
"pose_prior": {
|
| 207 |
+
"root": "assets/dposer_x"
|
| 208 |
+
}
|
| 209 |
+
}
|
| 210 |
+
},
|
| 211 |
+
"init": {
|
| 212 |
+
"model": {
|
| 213 |
+
"checkpoint_source_sha256": "6903c408027e994265c1bef3856c7554ef3b4d04eb04ae90a0066d07c39ee68f",
|
| 214 |
+
"class_name": "correction_regress_pose.CorrectionModel",
|
| 215 |
+
"stage": [
|
| 216 |
+
0,
|
| 217 |
+
1
|
| 218 |
+
],
|
| 219 |
+
"losses": [
|
| 220 |
+
1
|
| 221 |
+
],
|
| 222 |
+
"backbone": {
|
| 223 |
+
"frozen": [
|
| 224 |
+
0
|
| 225 |
+
],
|
| 226 |
+
"enc_channels": [
|
| 227 |
+
32,
|
| 228 |
+
64,
|
| 229 |
+
128,
|
| 230 |
+
256,
|
| 231 |
+
512
|
| 232 |
+
],
|
| 233 |
+
"dec_channels": [
|
| 234 |
+
256,
|
| 235 |
+
256,
|
| 236 |
+
256,
|
| 237 |
+
256
|
| 238 |
+
]
|
| 239 |
+
},
|
| 240 |
+
"add_noise": false,
|
| 241 |
+
"grid_size": [
|
| 242 |
+
0.01,
|
| 243 |
+
0.005,
|
| 244 |
+
0.001
|
| 245 |
+
],
|
| 246 |
+
"num_joints": 113,
|
| 247 |
+
"n_iters": 1,
|
| 248 |
+
"smplx_type": "smplh",
|
| 249 |
+
"smplx_path": "assets/body_models"
|
| 250 |
+
}
|
| 251 |
+
}
|
| 252 |
+
}
|
v-dataset/init.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e7746e2efb306e52b849dd1bc026c5a5afc090c6019451457b0e1f8238283a02
|
| 3 |
+
size 564966069
|
v-dataset/refine_solver.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:40a9fc631b131ac1f9b86bb83d72f3e9c37c779d1a63c285ee7eaa00cbf5d838
|
| 3 |
+
size 1233742899
|