Publish hftrainer artifact and model card

Mean.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26e136555dab04c94a129d446c26e6b9939cbf045fbf77bcf5462c1fb5a2001c
+size 1180

README.md

@@ -0,0 +1,98 @@
+---
+library_name: hftrainer
+pipeline_tag: other
+tags:
+- motion-generation
+- text-to-motion
+- humanml3d
+- motionlab
+- motion-editing
+license: other
+---
+
+<!-- This model card is synchronized from docs/model_zoo/motionlab.md by tools/sync_model_zoo_cards.py. -->
+
+# MotionLab - Unified Human Motion Generation and Editing
+
+Text-to-motion baseline integrated into the hftrainer Model Zoo. The runtime is
+self-contained under `hftrainer.models.motion.motionlab.network` and does not
+import the original repository at inference time.
+
+| | |
+|---|---|
+| **Task** | Text-to-Motion (T2M), motion generation / editing research stack |
+| **Bundle / Pipeline** | `MotionLabBundle` / `MotionLabPipeline` |
+| **Processed HF artifact** | [`ZeyuLing/hftrainer-motionlab-humanml3d`](https://huggingface.co/ZeyuLing/hftrainer-motionlab-humanml3d) |
+| **Motion representation** | **HumanML3D-263** (263-dim, 20 fps, 22 joints) |
+| **Architecture** | RFMotion / MotionFlow Transformer with CLIP text conditioning |
+| **Paper** | *MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigm*, Guo et al., ICCV 2025 - [arXiv:2502.02358](https://arxiv.org/abs/2502.02358) |
+| **Original code** | https://github.com/Diouo/MotionLab |
+
+---
+
+## Weights
+
+Self-contained hftrainer artifact:
+
+| Artifact | Location | Contents | Status |
+|---|---|---|---|
+| MotionLab HumanML3D | [`ZeyuLing/hftrainer-motionlab-humanml3d`](https://huggingface.co/ZeyuLing/hftrainer-motionlab-humanml3d) | `motionflow.ckpt` + `configs/` + `Mean.npy` / `Std.npy` + `mean_motion.npy` / `std_motion.npy` + `model_index.json` | public Hub artifact |
+| local mirror | `checkpoints/baselines/motionlab` | same layout | optional local cache |
+
+Use directly from the Hub:
+
+```python
+from hftrainer.pipelines.motionlab import MotionLabPipeline
+
+pipe = MotionLabPipeline.from_pretrained(
+    "ZeyuLing/hftrainer-motionlab-humanml3d",
+    device="cuda",
+)
+motions = pipe.infer_t2m(
+    ["a person walks forward then sits down"],
+    [120],
+)  # list of (T, 263)
+```
+
+For a local mirror:
+
+```python
+pipe = MotionLabPipeline.from_pretrained("checkpoints/baselines/motionlab", device="cuda")
+```
+
+## Motion Representation
+
+MotionLab natively generates **HumanML3D-263** at 20 fps. For shared SMPL and
+MotionStreamer-272 evaluation, use the validated bridge:
+
+```text
+HumanML3D-263 -> SMPL motion_135 via IK refine-80 -> MotionStreamer-272
+```
+
+The artifact contains both the HumanML3D denormalization statistics and
+MotionLab's internal motion statistics so the published pipeline does not depend
+on a separate dataset checkout.
+
+## HumanML3D Leaderboard Metrics
+
+The row below uses the shared HumanML3D official-test caption protocol and the
+HML263 round-trip GT reference for SMPL-based evaluators.
+
+| Evaluator | R1 up | R2 up | R3 up | FID down | MM down | Div up |
+|---|---:|---:|---:|---:|---:|---:|
+| MotionStreamer-272 | 0.6367 | 0.7882 | 0.8529 | 25.4469 | 17.9756 | 25.5355 |
+| MotionCLIP-135 no-L2 | 0.4807 | 0.6457 | 0.7353 | 102.7770 | 41.5472 | 23.0179 |
+
+Physical metrics:
+
+| Slide down | Float down | Jitter down | Dynamic down |
+|---:|---:|---:|---:|
+| 2.4231 | 4.0795 | 5.8493 | 24.3519 |
+
+## Implementation Notes
+
+- Artifact inference imports only `hftrainer.models.motion.motionlab.network`.
+- Config targets are rewritten from the original `rfmotion.*` namespace into the
+  vendored hftrainer namespace before model construction.
+- The default inference stage is `demo`, matching the validated qualitative
+  HumanML3D T2M setting.

Std.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6565a65ed9b31e23c328829a309e1c482be8b85fd23b43d65451a9b19a917f40
+size 1180

configs/assets.yaml

@@ -0,0 +1,50 @@
+FOLDER: './experiments'
+
+TEST:
+  FOLDER: './results'
+
+DATASET:
+  SMPL_PATH: './checkpoints/smpl/'
+  SMPLH_PATH: './checkpoints/smplh/'
+  WORD_VERTILIZER_PATH: './checkpoints/glove/'
+  HUMANML3D:
+    ROOT: './datasets/humanml3d/'
+    SPLIT_ROOT: './datasets/humanml3d/'
+  100STYLE:
+    ROOT: './datasets/100style/'
+    SPLIT_ROOT: './datasets/100style/'
+  HUMANML3D_100STYLE:
+    ROOT: './datasets/humanml3d_100style/'
+    SPLIT_ROOT: './datasets/humanml3d_100style/'
+  ALL:
+    ROOT: './datasets/all/'
+    SPLIT_ROOT: './datasets/all/'
+  MOTIONFIX_RETARGET:
+    ROOT: './datasets/motionfix_retarget/' 
+    SPLIT_ROOT: './datasets/motionfix_retarget/' 
+  MOTIONFIX:
+    ROOT: './datasets/motionfix/'
+    preproc:
+      stats_file: './datasets/motionfix/statistics_motionfix_feat3.npy' 
+      split_seed: 0
+      calculate_minmax: True
+      generate_joint_files: True
+      use_cuda: True
+      n_body_joints: 22
+      norm_type: std # norm or std
+    load_feats:
+        - "body_transl"
+        - "body_orient"
+        - "body_pose"
+        # - "body_transl_delta_pelv"
+        # - "body_orient_xy"
+        # - "z_orient_delta"
+        # - "body_pose"
+        # - "body_joints_local_wo_z_rot"
+
+model:
+  clip_path: './checkpoints/clip-vit-large-patch14' 
+  t5_path: './checkpoints/flan-t5-large'
+  t2m_path: './checkpoints/t2m/'
+  TMR_path: './checkpoints/TMR/'
+  style_encoder_path: './checkpoints/smoodi/style_encoder.pt'

configs/base.yaml

@@ -0,0 +1,141 @@
+# FOLDER: ./experiments
+SEED_VALUE: 42
+DEBUG: True
+TRAIN:
+  SPLIT: 'train'
+  NUM_WORKERS: 2 # Number of workers
+  BATCH_SIZE: 4 # Size of batches
+  START_EPOCH: 0 # Start epoch
+  END_EPOCH: 2000 # End epoch
+  RESUME: '' # Experiment path to be resumed training
+  PRETRAINED_VAE: ''
+  PRETRAINED: '' # Pretrained model path
+
+  OPTIM:
+    OPTIM.TYPE: 'AdamW' # Optimizer type
+    OPTIM.LR: 1e-4 # Learning rate
+
+  ABLATION:
+    VAE_TYPE: 'actor' # vae ablation: actor or mcross
+    VAE_ARCH: 'encoder_decoder' # mdiffusion vae architecture
+    PE_TYPE: 'actor' # mdiffusion mld or actor
+    DIFF_PE_TYPE: 'actor' # mdiffusion mld or actor
+    SKIP_CONNECT: False # skip connection for denoiser va
+    # use linear to expand mean and std rather expand token nums
+    MLP_DIST: False
+    IS_DIST: False # Mcross distribution kl
+    PREDICT_EPSILON: True # noise or motion
+
+EVAL:
+  SPLIT: 'gtest'
+  BATCH_SIZE: 1 # Evaluating Batch size
+  NUM_WORKERS: 12 # Evaluating Batch size
+
+TEST:
+  TEST_DIR: ''
+  CHECKPOINTS: '' # Pretrained model path
+  SPLIT: 'gtest'
+  BATCH_SIZE: 1 # Testing Batch size
+  NUM_WORKERS: 12 # Evaluating Batch size
+  SAVE_PREDICTIONS: False # Weather to save predictions
+  COUNT_TIME: False # Weather to count time during test
+  REPLICATION_TIMES: 5 # Number of times to replicate the test
+  MM_NUM_SAMPLES: 100 # Number of samples for multimodal test
+  MM_NUM_REPEATS: 30 # Number of repeats for multimodal test
+  MM_NUM_TIMES: 10 # Number of times to repeat the multimodal test
+  DIVERSITY_TIMES: 300 # Number of times to repeat the diversity test
+  REP_I: 0
+model:
+  target: 'modules'
+  t2m_textencoder:
+    dim_word: 300
+    dim_pos_ohot: 15
+    dim_text_hidden: 512
+    dim_coemb_hidden: 512
+
+  t2m_motionencoder:
+    dim_move_hidden: 512
+    dim_move_latent: 512
+    dim_motion_hidden: 1024
+    dim_motion_latent: 512
+LOSS:
+  LAMBDA_LATENT: 1e-5 # Lambda for latent losses
+  LAMBDA_KL: 1e-5 # Lambda for kl losses
+  LAMBDA_REC: 1.0 # Lambda for reconstruction losses
+  LAMBDA_JOINT: 1.0 # Lambda for joint losses
+  LAMBDA_GEN: 1.0 # Lambda for text-motion generation losses
+  LAMBDA_CROSS: 1.0 # Lambda for cross-reconstruction losses
+  LAMBDA_CYCLE: 1.0 # Lambda for cycle losses
+  LAMBDA_PRIOR: 0.0
+  DIST_SYNC_ON_STEP: True
+METRIC:
+  FORCE_IN_METER: True
+  DIST_SYNC_ON_STEP: True
+DATASET:
+  NCLASSES: 10
+  SAMPLER:
+    MAX_SQE: -1
+    MAX_LEN: 196
+    MIN_LEN: 40
+    MAX_TEXT_LEN: 20
+  KIT:
+    PICK_ONE_TEXT: true
+    FRAME_RATE: 12.5
+    UNIT_LEN: 4
+  HUMANML3D:
+    PICK_ONE_TEXT: true
+    FRAME_RATE: 20.0
+    UNIT_LEN: 4
+  MOTIONFIX_RETARGET:
+    PICK_ONE_TEXT: true
+    FRAME_RATE: 20.0
+    UNIT_LEN: 4
+  HUMANML3D_100STYLE:
+    PICK_ONE_TEXT: true
+    FRAME_RATE: 20.0
+    UNIT_LEN: 4
+  ALL:
+    PICK_ONE_TEXT: true
+    FRAME_RATE: 20.0
+    UNIT_LEN: 4
+  HUMANACT12:
+    NUM_FRAMES: 60
+    POSE_REP: rot6d
+    GLOB: true
+    TRANSLATION: true
+  UESTC:
+    NUM_FRAMES: 60
+    POSE_REP: rot6d
+    GLOB: true
+    TRANSLATION: true
+LOGGER:
+  SACE_CHECKPOINT_EPOCH: 1
+  LOG_EVERY_STEPS: 1
+  VAL_EVERY_STEPS: 10
+  TENSORBOARD: true
+  WANDB:
+    OFFLINE: false
+    PROJECT: null
+    RESUME_ID: null
+RENDER:
+  JOINT_TYPE: mmm
+  INPUT_MODE: npy
+  DIR: ''
+  NPY: ''
+  DENOISING: true
+  OLDRENDER: true
+  RES: high
+  DOWNSAMPLE: true
+  FPS: 12.5
+  CANONICALIZE: true
+  EXACT_FRAME: 0.5
+  NUM: 7
+  MODE: sequence
+  VID_EXT: mp4
+  ALWAYS_ON_FLOOR: false
+  GT: false
+DEMO:
+  MOTION_TRANSFER: false
+  RENDER: false
+  FRAME_RATE: 12.5
+  EXAMPLE: null

configs/config_rfmotion.yaml

@@ -0,0 +1,137 @@
+####################################
+# The following are general settings
+####################################
+
+# Experiment name, more details in Section 'Experiment Name Explanation'
+NAME: baseline
+# 1. use a tiny dataset for trianing and evaluation
+# 2. validate more intensively
+# 3. will not use `wandb logger`
+DEBUG: False
+# Devices. Optional: “cpu”, “gpu”
+ACCELERATOR: 'gpu'
+# Index of GPUs eg. [0] or [0,1,2,3]
+DEVICE: [0]
+
+#####################################
+# The following are training settings
+#####################################
+TRAIN:
+  # Model stage. Optional: "vae", "diffusion"
+  STAGE: diffusion
+  # Training dataset name
+  DATASETS: ['all'] 
+  # Number of dataloader workers
+  NUM_WORKERS: 8
+  # Size of batches
+  BATCH_SIZE: 64
+  # Total epochs for training
+  END_EPOCH: 8000
+
+  RESUME: '' # Resume training from this path
+  PRETRAINED_VAE: '' # vae model path
+  OPTIM:
+    TYPE: AdamW # Optimizer type
+    LR: 1e-4 # Learning rate
+  # Ablation study configurations.
+  ABLATION:
+    VAE: False
+    VAE_PE_TYPE: "sine"
+    VAE_PE_DIM: "1D"
+    VAE_LATENT_NUMS: 8
+
+    RF_SEP: False
+    LN_Sampling: False
+    WARM_UP: False
+    HINT_GUIDANCE: False
+
+#####################################
+# The following are basic model settings
+#####################################
+model:
+  model_type: 'rfmotion'
+  condition_type: 'all'
+  token_dim: 512
+
+  # ## For All
+  text_guidance_scale: 5.75
+  hint_guidance_scale: 1.5
+  text_hint_guidance_scale: 1.75
+  inbetween_guidance_scale: 1.5
+  text_inbetween_guidance_scale: 1.75
+  source_text_guidance_scale_1: 2.25
+  source_text_guidance_scale_2: 2.25
+  source_hint_guidance_scale_1: 2.25  
+  source_hint_guidance_scale_2: 2.25
+  source_text_hint_guidance_scale_1: 2
+  source_text_hint_guidance_scale_2: 2
+  style_guidance_scale: 1.5
+  drop_style_guidance_prob: 0.2
+  drop_content_guidance_prob: 0.2
+  
+#####################################
+# The following are validation settings
+#####################################
+EVAL:
+  DATASETS: ['all'] # Evaluating datasets
+  BATCH_SIZE: 32 # Evaluating Batch size
+  SPLIT: test
+
+#####################################
+# The following are testing settings
+#####################################
+TEST:
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['all']
+  SPLIT: test
+  BATCH_SIZE: 32
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+
+#####################################
+# The following are demo settings
+#####################################
+DEMO:
+  TYPE: ""
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['all']
+  SPLIT: test
+  BATCH_SIZE: 1
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+  SAMPLE_NUMS: 10
+  REPLICATION: 1
+
+#####################################
+# The following are metric settings
+#####################################
+METRIC:
+  TYPE: ["MaskedMetrics", "TM2TMetrics", "SourceTextMetrics", "SourceHintMetrics", "SourceTextHintMetrics", "InbetweenMetrics", "TextInbetweenMetrics","TextHintMetrics", "HintMetrics", "StyleMetrics", ]
+
+#####################################
+# The following are training losses settings
+#####################################
+LOSS:
+  TYPE: mld # Losses type
+  LAMBDA_LATENT: 1.0e-5 # Lambda for latent Losses
+  LAMBDA_KL: 1.0e-4 # Lambda for kl Losses
+  LAMBDA_REC: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_GEN: 1.0 # Lambda for text-motion generation losses
+  LAMBDA_CROSS: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_CYCLE: 0.0 # Lambda for cycle Losses
+  LAMBDA_PRIOR: 0.0
+  DIST_SYNC_ON_STEP: False # Sync Losses on step when distributed trained
+
+#####################################
+# The following are loggers settings
+#####################################
+LOGGER:
+  SAVE_CHECKPOINT_EPOCH: 50
+  CHECK_VAL_EVERY_N_EPOCH: 25
+  TENSORBOARD: True
+  WANDB:
+    PROJECT: null
+    OFFLINE: False
+    RESUME_ID: null

configs/config_rfmotion_hint.yaml

@@ -0,0 +1,128 @@
+####################################
+# The following are general settings
+####################################
+
+# Experiment name, more details in Section 'Experiment Name Explanation'
+NAME: baseline_Hint
+# 1. use a tiny dataset for trianing and evaluation
+# 2. validate more intensively
+# 3. will not use `wandb logger`
+DEBUG: False
+# Devices. Optional: “cpu”, “gpu”
+ACCELERATOR: 'gpu'
+# Index of GPUs eg. [0] or [0,1,2,3]
+DEVICE: [0]
+
+#####################################
+# The following are training settings
+#####################################
+TRAIN:
+  # Model stage. Optional: "vae", "diffusion"
+  STAGE: diffusion
+  # Training dataset name
+  DATASETS: ['humanml3d'] # ['humanml3d'], ['motionfix'] ['motionfix_retarget'] ['100style'] ['humanml3d_100style'] ['all']
+  # Number of dataloader workers
+  NUM_WORKERS: 8
+  # Size of batches
+  BATCH_SIZE: 64
+  # Total epochs for training
+  END_EPOCH: 8000
+
+  RESUME: '' # Resume training from this path
+  PRETRAINED_VAE: '' # vae model path
+  OPTIM:
+    TYPE: AdamW # Optimizer type
+    LR: 1e-4 # Learning rate
+  # Ablation study configurations.
+  ABLATION:
+    VAE: False
+    VAE_PE_TYPE: "sine"
+    VAE_PE_DIM: "1D"
+    VAE_LATENT_NUMS: 8
+
+    RF_SEP: False
+    LN_Sampling: False
+    WARM_UP: False
+    HINT_GUIDANCE: False
+
+#####################################
+# The following are basic model settings
+#####################################
+model:
+  model_type: 'rfmotion_seperate'
+  condition_type: 'hint'
+  token_dim: 512
+
+  ## For Hint
+  text_hint_guidance_scale: 2
+  text_hint_guidance_prob: 0.7
+  text_guidance_prob: 0.1
+  hint_guidance_prob: 0.1
+  none_guidance_prob: 0.1
+
+#####################################
+# The following are validation settings
+#####################################
+EVAL:
+  DATASETS: ['humanml3d'] # Evaluating datasets
+  BATCH_SIZE: 32 # Evaluating Batch size
+  SPLIT: test
+
+#####################################
+# The following are testing settings
+#####################################
+TEST:
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['humanml3d']
+  SPLIT: test
+  BATCH_SIZE: 32
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+
+#####################################
+# The following are demo settings
+#####################################
+DEMO:
+  TYPE: "hint"
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['humanml3d']
+  SPLIT: test
+  BATCH_SIZE: 1
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+  SAMPLE_NUMS: 10
+  REPLICATION: 1
+
+#####################################
+# The following are metric settings
+#####################################
+METRIC:
+  TYPE: ["TextHintMetrics"]
+
+#####################################
+# The following are training losses settings
+#####################################
+LOSS:
+  TYPE: mld # Losses type
+  LAMBDA_LATENT: 1.0e-5 # Lambda for latent Losses
+  LAMBDA_KL: 1.0e-4 # Lambda for kl Losses
+  LAMBDA_REC: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_GEN: 1.0 # Lambda for text-motion generation losses
+  LAMBDA_CROSS: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_CYCLE: 0.0 # Lambda for cycle Losses
+  LAMBDA_PRIOR: 0.0
+  DIST_SYNC_ON_STEP: False # Sync Losses on step when distributed trained
+
+#####################################
+# The following are loggers settings
+#####################################
+LOGGER:
+  SAVE_CHECKPOINT_EPOCH: 50
+  CHECK_VAL_EVERY_N_EPOCH: 10
+  TENSORBOARD: True
+  WANDB:
+    PROJECT: null
+    OFFLINE: False
+    RESUME_ID: null

configs/config_rfmotion_inbetween.yaml

@@ -0,0 +1,124 @@
+####################################
+# The following are general settings
+####################################
+
+# Experiment name, more details in Section 'Experiment Name Explanation'
+NAME: baseline_Inbetween
+# 1. use a tiny dataset for trianing and evaluation
+# 2. validate more intensively
+# 3. will not use `wandb logger`
+DEBUG: False
+# Devices. Optional: “cpu”, “gpu”
+ACCELERATOR: 'gpu'
+# Index of GPUs eg. [0] or [0,1,2,3]
+DEVICE: [0]
+
+#####################################
+# The following are training settings
+#####################################
+TRAIN:
+  # Model stage. Optional: "vae", "diffusion"
+  STAGE: diffusion
+  # Training dataset name
+  DATASETS: ['humanml3d'] # ['humanml3d'], ['motionfix'] ['motionfix_retarget'] ['100style'] ['humanml3d_100style'] ['all']
+  # Number of dataloader workers
+  NUM_WORKERS: 8
+  # Size of batches
+  BATCH_SIZE: 64
+  # Total epochs for training
+  END_EPOCH: 8000
+
+  RESUME: '' # Resume training from this path
+  PRETRAINED_VAE: '' # vae model path
+  OPTIM:
+    TYPE: AdamW # Optimizer type
+    LR: 1e-4 # Learning rate
+  # Ablation study configurations.
+  ABLATION:
+    VAE: False
+    VAE_PE_TYPE: "sine"
+    VAE_PE_DIM: "1D"
+    VAE_LATENT_NUMS: 8
+
+    RF_SEP: False
+    LN_Sampling: False
+    WARM_UP: False
+    HINT_GUIDANCE: False
+
+#####################################
+# The following are basic model settings
+#####################################
+model:
+  model_type: 'rfmotion_seperate'
+  condition_type: 'inbetween'
+  token_dim: 512
+
+  ## For Inbetween
+  inbetween_guidance_scale: 1.5
+
+#####################################
+# The following are validation settings
+#####################################
+EVAL:
+  DATASETS: ['humanml3d'] # Evaluating datasets
+  BATCH_SIZE: 32 # Evaluating Batch size
+  SPLIT: test
+
+#####################################
+# The following are testing settings
+#####################################
+TEST:
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['humanml3d']
+  SPLIT: test
+  BATCH_SIZE: 32
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+
+#####################################
+# The following are demo settings
+#####################################
+DEMO:
+  TYPE: "inbetween" 
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['humanml3d']
+  SPLIT: test
+  BATCH_SIZE: 1
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+  SAMPLE_NUMS: 10
+  REPLICATION: 1
+
+#####################################
+# The following are metric settings
+#####################################
+METRIC:
+  TYPE: ["TextInbetweenMetrics"] # UncondMetrics, ReconMetrics, TM2TMetrics, MMMetrics, SourceTextMetrics, SourceHintMetrics, InbetweenMetrics, HintMetrics, StyleMetrics
+
+#####################################
+# The following are training losses settings
+#####################################
+LOSS:
+  TYPE: mld # Losses type
+  LAMBDA_LATENT: 1.0e-5 # Lambda for latent Losses
+  LAMBDA_KL: 1.0e-4 # Lambda for kl Losses
+  LAMBDA_REC: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_GEN: 1.0 # Lambda for text-motion generation losses
+  LAMBDA_CROSS: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_CYCLE: 0.0 # Lambda for cycle Losses
+  LAMBDA_PRIOR: 0.0
+  DIST_SYNC_ON_STEP: False # Sync Losses on step when distributed trained
+
+#####################################
+# The following are loggers settings
+#####################################
+LOGGER:
+  SAVE_CHECKPOINT_EPOCH: 50
+  CHECK_VAL_EVERY_N_EPOCH: 10
+  TENSORBOARD: True
+  WANDB:
+    PROJECT: null
+    OFFLINE: False
+    RESUME_ID: null

configs/config_rfmotion_sourcehint.yaml

@@ -0,0 +1,129 @@
+####################################
+# The following are general settings
+####################################
+
+# Experiment name, more details in Section 'Experiment Name Explanation'
+NAME: baseline_SourceHint
+# 1. use a tiny dataset for trianing and evaluation
+# 2. validate more intensively
+# 3. will not use `wandb logger`
+DEBUG: False
+# Devices. Optional: “cpu”, “gpu”
+ACCELERATOR: 'gpu'
+# Index of GPUs eg. [0] or [0,1,2,3]
+DEVICE: [0]
+
+#####################################
+# The following are training settings
+#####################################
+TRAIN:
+  # Model stage. Optional: "vae", "diffusion"
+  STAGE: diffusion
+  # Training dataset name
+  DATASETS: ['motionfix_retarget'] # ['humanml3d'], ['motionfix'] ['motionfix_retarget'] ['100style'] ['humanml3d_100style'] ['all']
+  # Number of dataloader workers
+  NUM_WORKERS: 8
+  # Size of batches
+  BATCH_SIZE: 64
+  # Total epochs for training
+  END_EPOCH: 8000
+
+  RESUME: '' # Resume training from this path
+  PRETRAINED_VAE: '' # vae model path
+  OPTIM:
+    TYPE: AdamW # Optimizer type
+    LR: 1e-4 # Learning rate
+  # Ablation study configurations.
+  ABLATION:
+    VAE: False
+    VAE_PE_TYPE: "sine"
+    VAE_PE_DIM: "1D"
+    VAE_LATENT_NUMS: 8
+
+    RF_SEP: False
+    LN_Sampling: False
+    WARM_UP: False
+    HINT_GUIDANCE: False
+
+#####################################
+# The following are basic model settings
+#####################################
+model:
+  model_type: 'rfmotion_seperate'
+  condition_type: 'source_hint'
+  token_dim: 512
+
+  ## For SourceHint
+  source_hint_guidance_scale_1: 2
+  source_hint_guidance_scale_2: 2
+  source_hint_guidance_prob: 0.7
+  hint_guidance_prob: 0.1
+  source_guidance_prob: 0.1
+  none_guidance_prob: 0.1
+
+#####################################
+# The following are validation settings
+#####################################
+EVAL:
+  DATASETS: ['motionfix_retarget'] # Evaluating datasets
+  BATCH_SIZE: 32 # Evaluating Batch size
+  SPLIT: test
+
+#####################################
+# The following are testing settings
+#####################################
+TEST:
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['motionfix_retarget']
+  SPLIT: test
+  BATCH_SIZE: 32
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+
+#####################################
+# The following are demo settings
+#####################################
+DEMO:
+  TYPE: "source_hint" 
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['motionfix_retarget']
+  SPLIT: test
+  BATCH_SIZE: 1
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+  SAMPLE_NUMS: 10
+  REPLICATION: 1
+
+#####################################
+# The following are metric settings
+#####################################
+METRIC:
+  TYPE: ["SourceHintMetrics"]
+
+#####################################
+# The following are training losses settings
+#####################################
+LOSS:
+  TYPE: mld # Losses type
+  LAMBDA_LATENT: 1.0e-5 # Lambda for latent Losses
+  LAMBDA_KL: 1.0e-4 # Lambda for kl Losses
+  LAMBDA_REC: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_GEN: 1.0 # Lambda for text-motion generation losses
+  LAMBDA_CROSS: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_CYCLE: 0.0 # Lambda for cycle Losses
+  LAMBDA_PRIOR: 0.0
+  DIST_SYNC_ON_STEP: False # Sync Losses on step when distributed trained
+
+#####################################
+# The following are loggers settings
+#####################################
+LOGGER:
+  SAVE_CHECKPOINT_EPOCH: 50
+  CHECK_VAL_EVERY_N_EPOCH: 10
+  TENSORBOARD: True
+  WANDB:
+    PROJECT: null
+    OFFLINE: False
+    RESUME_ID: null

configs/config_rfmotion_sourcetext.yaml

@@ -0,0 +1,128 @@
+####################################
+# The following are general settings
+####################################
+
+# Experiment name, more details in Section 'Experiment Name Explanation'
+NAME: baseline_SourceText
+# 1. use a tiny dataset for trianing and evaluation
+# 2. validate more intensively
+# 3. will not use `wandb logger`
+DEBUG: False
+# Devices. Optional: “cpu”, “gpu”
+ACCELERATOR: 'gpu'
+# Index of GPUs eg. [0] or [0,1,2,3]
+DEVICE: [0]
+
+#####################################
+# The following are training settings
+#####################################
+TRAIN:
+  # Model stage. Optional: "vae", "diffusion"
+  STAGE: diffusion
+  # Training dataset name
+  DATASETS: ['motionfix_retarget'] # ['humanml3d'], ['motionfix'] ['motionfix_retarget'] ['100style'] ['humanml3d_100style'] ['all']
+  # Number of dataloader workers
+  NUM_WORKERS: 8
+  # Size of batches
+  BATCH_SIZE: 64
+  # Total epochs for training
+  END_EPOCH: 8000
+
+  RESUME: '' # Resume training from this path
+  PRETRAINED_VAE: '' # vae model path
+  OPTIM:
+    TYPE: AdamW # Optimizer type
+    LR: 1e-4 # Learning rate
+  # Ablation study configurations.
+  ABLATION:
+    VAE: False
+    VAE_PE_TYPE: "sine"
+    VAE_PE_DIM: "1D"
+    VAE_LATENT_NUMS: 8
+
+    RF_SEP: False
+    LN_Sampling: False
+    WARM_UP: False
+    HINT_GUIDANCE: False
+
+#####################################
+# The following are basic model settings
+#####################################
+model:
+  model_type: 'rfmotion_seperate'
+  condition_type: 'source_text'
+  token_dim: 512
+
+  ## For SourceText
+  source_text_guidance_scale_1: 2 
+  source_text_guidance_scale_2: 2
+  source_text_guidance_prob: 0.8
+  source_guidance_prob: 0.1
+  none_guidance_prob: 0.1
+
+#####################################
+# The following are validation settings
+#####################################
+EVAL:
+  DATASETS: ['motionfix_retarget'] # Evaluating datasets
+  BATCH_SIZE: 32 # Evaluating Batch size
+  SPLIT: test
+
+#####################################
+# The following are testing settings
+#####################################
+TEST:
+  TYPE: "source_text" 
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['motionfix_retarget']
+  SPLIT: test
+  BATCH_SIZE: 32
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+
+#####################################
+# The following are demo settings
+#####################################
+DEMO:
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['motionfix_retarget']
+  SPLIT: test
+  BATCH_SIZE: 1
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+  SAMPLE_NUMS: 10
+  REPLICATION: 1
+
+#####################################
+# The following are metric settings
+#####################################
+METRIC:
+  TYPE: ["SourceTextMetrics"]
+
+#####################################
+# The following are training losses settings
+#####################################
+LOSS:
+  TYPE: mld # Losses type
+  LAMBDA_LATENT: 1.0e-5 # Lambda for latent Losses
+  LAMBDA_KL: 1.0e-4 # Lambda for kl Losses
+  LAMBDA_REC: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_GEN: 1.0 # Lambda for text-motion generation losses
+  LAMBDA_CROSS: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_CYCLE: 0.0 # Lambda for cycle Losses
+  LAMBDA_PRIOR: 0.0
+  DIST_SYNC_ON_STEP: False # Sync Losses on step when distributed trained
+
+#####################################
+# The following are loggers settings
+#####################################
+LOGGER:
+  SAVE_CHECKPOINT_EPOCH: 50
+  CHECK_VAL_EVERY_N_EPOCH: 10
+  TENSORBOARD: True
+  WANDB:
+    PROJECT: null
+    OFFLINE: False
+    RESUME_ID: null

configs/config_rfmotion_style.yaml

@@ -0,0 +1,130 @@
+####################################
+# The following are general settings
+####################################
+
+# Experiment name, more details in Section 'Experiment Name Explanation'
+NAME: baseline_Style
+# 1. use a tiny dataset for trianing and evaluation
+# 2. validate more intensively
+# 3. will not use `wandb logger`
+DEBUG: False
+# Devices. Optional: “cpu”, “gpu”
+ACCELERATOR: 'gpu'
+# Index of GPUs eg. [0] or [0,1,2,3]
+DEVICE: [0]
+
+#####################################
+# The following are training settings
+#####################################
+TRAIN:
+  # Model stage. Optional: "vae", "diffusion"
+  STAGE: diffusion
+  # Training dataset name
+  DATASETS: ['humanml3d'] # ['humanml3d'], ['motionfix'] ['motionfix_retarget'] ['100style'] ['humanml3d_100style'] ['all']
+  # Number of dataloader workers
+  NUM_WORKERS: 8
+  # Size of batches
+  BATCH_SIZE: 64
+  # Total epochs for training
+  END_EPOCH: 8000
+
+  RESUME: '' # Resume training from this path
+  PRETRAINED_VAE: '' # vae model path
+  OPTIM:
+    TYPE: AdamW # Optimizer type
+    LR: 1e-4 # Learning rate
+  # Ablation study configurations.
+  ABLATION:
+    VAE: False
+    VAE_PE_TYPE: "sine"
+    VAE_PE_DIM: "1D"
+    VAE_LATENT_NUMS: 8
+
+    RF_SEP: False
+    LN_Sampling: False
+    WARM_UP: False
+    HINT_GUIDANCE: False
+
+#####################################
+# The following are basic model settings
+#####################################
+model:
+  model_type: 'rfmotion_seperate'
+  condition_type: 'style'
+  token_dim: 512
+
+  # ## For Style
+  style_guidance_scale: 2.5
+
+  all_guidance_prob: 0.6
+  drop_content_guidance_prob: 0.1
+  drop_hint_guidance_prob: 0.1
+  drop_style_guidance_prob: 0.1
+  none_guidance_prob: 0.1
+
+#####################################
+# The following are validation settings
+#####################################
+EVAL:
+  DATASETS: ['humanml3d'] # Evaluating datasets
+  BATCH_SIZE: 32 # Evaluating Batch size
+  SPLIT: test
+
+#####################################
+# The following are testing settings
+#####################################
+TEST:
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['humanml3d']
+  SPLIT: test
+  BATCH_SIZE: 32
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+
+#####################################
+# The following are demo settings
+#####################################
+DEMO:
+  TYPE: "style" 
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['humanml3d']
+  SPLIT: test
+  BATCH_SIZE: 1
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+  SAMPLE_NUMS: 10
+  REPLICATION: 1
+
+#####################################
+# The following are metric settings
+#####################################
+METRIC:
+  TYPE: ["StyleMetrics"] # UncondMetrics, ReconMetrics, TM2TMetrics, MMMetrics, SourceTextMetrics, SourceHintMetrics, InbetweenMetrics, HintMetrics, StyleMetrics
+
+#####################################
+# The following are training losses settings
+#####################################
+LOSS:
+  TYPE: mld # Losses type
+  LAMBDA_LATENT: 1.0e-5 # Lambda for latent Losses
+  LAMBDA_KL: 1.0e-4 # Lambda for kl Losses
+  LAMBDA_REC: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_GEN: 1.0 # Lambda for text-motion generation losses
+  LAMBDA_CROSS: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_CYCLE: 0.0 # Lambda for cycle Losses
+  LAMBDA_PRIOR: 0.0
+  DIST_SYNC_ON_STEP: False # Sync Losses on step when distributed trained
+
+#####################################
+# The following are loggers settings
+#####################################
+LOGGER:
+  SAVE_CHECKPOINT_EPOCH: 50
+  CHECK_VAL_EVERY_N_EPOCH: 10
+  TENSORBOARD: True
+  WANDB:
+    PROJECT: null
+    OFFLINE: False
+    RESUME_ID: null

configs/config_rfmotion_text.yaml

@@ -0,0 +1,126 @@
+####################################
+# The following are general settings
+####################################
+
+# Experiment name, more details in Section 'Experiment Name Explanation'
+NAME: baseline_Text
+# 1. use a tiny dataset for trianing and evaluation
+# 2. validate more intensively
+# 3. will not use `wandb logger`
+DEBUG: False
+# Devices. Optional: “cpu”, “gpu”
+ACCELERATOR: 'gpu'
+# Index of GPUs eg. [0] or [0,1,2,3]
+DEVICE: [0]
+
+#####################################
+# The following are training settings
+#####################################
+TRAIN:
+  # Model stage. Optional: "vae", "diffusion"
+  STAGE: diffusion
+  # Training dataset name
+  DATASETS: ['humanml3d'] # ['humanml3d'], ['motionfix'] ['motionfix_retarget'] ['100style'] ['humanml3d_100style'] ['all']
+  # Number of dataloader workers
+  NUM_WORKERS: 8
+  # Size of batches
+  BATCH_SIZE: 64
+  # Total epochs for training
+  END_EPOCH: 8000
+
+  RESUME: '' # Resume training from this path
+  PRETRAINED_VAE: '' # vae model path
+  OPTIM:
+    TYPE: AdamW # Optimizer type
+    LR: 1e-4 # Learning rate
+  # Ablation study configurations.
+  ABLATION:
+    VAE: False
+    VAE_PE_TYPE: "sine"
+    VAE_PE_DIM: "1D"
+    VAE_LATENT_NUMS: 8
+
+    RF_SEP: False
+    LN_Sampling: False
+    WARM_UP: False
+    HINT_GUIDANCE: False
+
+#####################################
+# The following are basic model settings
+#####################################
+model:
+  model_type: 'rfmotion_seperate'
+  condition_type: 'text'
+  token_dim: 512
+
+  ## For Text
+  text_guidance_scale: 2.5
+  text_guidance_prob: 0.9 
+  none_guidance_prob: 0.1
+
+#####################################
+# The following are validation settings
+#####################################
+EVAL:
+  DATASETS: ['humanml3d'] # Evaluating datasets
+  BATCH_SIZE: 32 # Evaluating Batch size
+  SPLIT: test
+
+#####################################
+# The following are testing settings
+#####################################
+TEST:
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['humanml3d']
+  SPLIT: test
+  BATCH_SIZE: 32
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+
+#####################################
+# The following are demo settings
+#####################################
+DEMO:
+  TYPE: "text" 
+  CHECKPOINTS: ""  # Pretrained model path
+  DATASETS: ['humanml3d']
+  SPLIT: test
+  BATCH_SIZE: 1
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+  SAMPLE_NUMS: 10
+  REPLICATION: 1
+
+#####################################
+# The following are metric settings
+#####################################
+METRIC:
+  TYPE: ["TM2TMetrics"] # MMMetrics
+
+#####################################
+# The following are training losses settings
+#####################################
+LOSS:
+  TYPE: mld # Losses type
+  LAMBDA_LATENT: 1.0e-5 # Lambda for latent Losses
+  LAMBDA_KL: 1.0e-4 # Lambda for kl Losses
+  LAMBDA_REC: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_GEN: 1.0 # Lambda for text-motion generation losses
+  LAMBDA_CROSS: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_CYCLE: 0.0 # Lambda for cycle Losses
+  LAMBDA_PRIOR: 0.0
+  DIST_SYNC_ON_STEP: False # Sync Losses on step when distributed trained
+
+#####################################
+# The following are loggers settings
+#####################################
+LOGGER:
+  SAVE_CHECKPOINT_EPOCH: 50
+  CHECK_VAL_EVERY_N_EPOCH: 10
+  TENSORBOARD: True
+  WANDB:
+    PROJECT: null
+    OFFLINE: False
+    RESUME_ID: null

configs/config_vae.yaml

@@ -0,0 +1,102 @@
+NAME: baseline_Feat4_Divide8-256_NativeROPE # Experiment name
+DEBUG: False # Debug mode
+ACCELERATOR: 'gpu' # Devices optioncal: “cpu”, “gpu”, “tpu”, “ipu”, “hpu”, “mps, “auto”
+DEVICE: [0] # Index of gpus eg. [0] or [0,1,2,3]
+
+# Training configuration
+TRAIN:
+  #---------------------------------
+  STAGE: vae # stage "vae" or "diffusion", "vae_diffusion"
+  #---------------------------------
+  DATASETS: ['motionfix'] # Training datasets
+  NUM_WORKERS: 8 # Number of workers
+  BATCH_SIZE: 128 # 128 # Size of batches
+  START_EPOCH: 0 # Start epoch
+  END_EPOCH: 4000 # End epoch
+  RESUME: '' # Resume training from this path
+  PRETRAINED_VAE: ''
+  OPTIM:
+    TYPE: AdamW # Optimizer type
+    LR: 1e-4 # Learning rate
+
+  ABLATION:
+    VAE: True
+    VAE_PE_TYPE: "rope"
+    VAE_PE_DIM: "1D"
+
+    RF_PE_TYPE: "sine"
+    RF_PE_DIM: "1D"
+    RF_TIMEEMBED: "time" # "time_text"
+    RF_SEP: False
+    LN_Sampling: False
+    WARM_UP: False
+    HINT_GUIDANCE: False
+
+# Model Configuration
+model:
+  model_type: "vae"
+  latent_dim: [8, 256]
+
+  mixing_scale: 1
+  guidance_type: 'source_text'
+
+  ## For SourceText
+  text_guidance_scale: 2 
+  text_guidance_prob: 0.85 
+  source_guidance_scale: 2
+  source_guidance_prob: 0.075
+  none_guidance_prob: 0.075
+
+  all_guidance_scale: 1.0
+  all_guidance_prob: 0.7
+  hint_guidance_scale: 2
+  hint_guidance_prob: 0.1
+
+  # ## For SourceHint
+  # all_guidance_scale: 1.0
+  # all_guidance_prob: 0.7
+  # hint_guidance_scale: 2
+  # hint_guidance_prob: 0.1
+  # source_guidance_scale: 2
+  # source_guidance_prob: 0.1
+  # none_guidance_prob: 0.1
+
+  # text_guidance_scale: 2 
+  # text_guidance_prob: 0.85 
+
+# Evaluating Configuration
+EVAL:
+  DATASETS: ['motionfix'] # Evaluating datasets
+  BATCH_SIZE: 32 # 32 # Evaluating Batch size
+  SPLIT: test
+
+# Test Configuration
+TEST:
+  CHECKPOINTS: '' # Pretrained model path
+  DATASETS: ['motionfix'] # training datasets
+  SPLIT: test
+  BATCH_SIZE: 32 # 32 # training Batch size
+  MEAN: False
+  NUM_SAMPLES: 1
+  FACT: 1
+
+# Metric Configuration
+METRIC:
+  TYPE: ['MRMetrics','UncondMetrics']  # TYPE: ['MRMetrics','UncondMetrics'] # TYPE: ['MRMetrics'] 
+
+# Losses Configuration
+LOSS:
+  LAMBDA_LATENT: 1.0e-5 # Lambda for latent Losses
+  LAMBDA_KL: 1.0e-4 # Lambda for kl Losses
+  LAMBDA_REC: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_GEN: 1.0 # Lambda for text-motion generation losses
+  LAMBDA_CROSS: 1.0 # Lambda for reconstruction Losses
+  LAMBDA_CYCLE: 0.0 # Lambda for cycle Losses
+  LAMBDA_PRIOR: 0.0
+  DIST_SYNC_ON_STEP: False # Sync Losses on step when distributed trained
+
+# Logger configuration
+LOGGER:
+  SAVE_CHECKPOINT_EPOCH: 50
+  CHECK_VAL_EVERY_N_EPOCH: 5
+  TENSORBOARD: True

configs/modules/content_encoder.yaml

@@ -0,0 +1,18 @@
+content_encoder:
+  # Optional: mld_vae, vposert_vae
+  target: hftrainer.models.motion.motionlab.network.rfmotion.models.architectures.content_encoder.MldVae
+  params:
+    arch: 'encoder_decoder'
+    ff_size: 1024
+    num_layers: 9
+    num_heads: 4
+    dropout: 0.1
+    normalize_before: false
+    activation: 'gelu'
+    position_embedding: 'learned'
+    latent_dim: [7, 256]
+    nfeats: 263
+    ablation: 
+      SKIP_CONNECT: True
+      PE_TYPE: mld
+      DIFF_PE_TYPE: mld

configs/modules/denoiser.yaml

@@ -0,0 +1,18 @@
+denoiser:
+  target: hftrainer.models.motion.motionlab.network.rfmotion.models.architectures.rfmotion_denoiser_4path.RFMotionDenoiser # rfmotion_denoiser.RFMotionDenoiser
+  params:
+    text_encoded_dim: 768
+    ff_size: 1024
+    num_layers: 9
+    num_heads: 8
+    dropout: 0.1
+    normalize_before: False
+    activation: 'gelu'
+    flip_sin_to_cos: True
+    return_intermediate_dec: False
+    arch: trans_enc
+    freq_shift: 0
+    token_dim: ${model.token_dim}
+    nfeats: ${DATASET.NFEATS}
+    nclasses: ${DATASET.NCLASSES}
+    ablation: ${TRAIN.ABLATION}

configs/modules/evaluators.yaml

@@ -0,0 +1,20 @@
+t2m_textencoder:
+  target: hftrainer.models.motion.motionlab.network.rfmotion.models.architectures.t2m_textenc.TextEncoderBiGRUCo
+  params:
+    word_size: 300
+    pos_size: 15
+    hidden_size: 512
+    output_size: 512
+
+t2m_moveencoder:
+  target: hftrainer.models.motion.motionlab.network.rfmotion.models.architectures.t2m_textenc.MovementConvEncoder
+  params:
+    hidden_size: 512
+    output_size: 512
+
+t2m_motionencoder:
+  target: hftrainer.models.motion.motionlab.network.rfmotion.models.architectures.t2m_motionenc.MotionEncoder
+  params:
+    input_size: ${model.t2m_moveencoder.output_size}
+    hidden_size: 1024
+    output_size: 512

configs/modules/motion_vae.yaml

@@ -0,0 +1,12 @@
+motion_vae:
+  target: hftrainer.models.motion.motionlab.network.rfmotion.models.architectures.rfmotion_vae_nativerope.MldVae # rfmotion_vae.RFMotionVae # mld_vae.MldVae
+  params:
+    ff_size: 1024
+    num_layers: 9
+    num_heads: 8
+    dropout: 0.1
+    normalize_before: false
+    activation: 'gelu'
+    latent_dim: ${model.latent_dim}
+    nfeats: ${DATASET.NFEATS}
+    ablation: ${TRAIN.ABLATION}

configs/modules/motionclip_params_263.yaml

@@ -0,0 +1,62 @@
+activation: gelu
+archiname: transformer
+batch_size: 128
+clip_image_losses:
+- cosine
+clip_lambda_ce: 1.0
+clip_lambda_cosine: 1.0
+clip_lambda_mse: 1.0
+clip_lambdas:
+  image:
+    cosine: 1.0
+  text:
+    cosine: 1.0
+clip_layers: 12
+clip_text_losses:
+- cosine
+clip_training: ''
+cuda: true
+datapath: ./data/amass_xyz_911_feats_notrans/amass_30fps_db.pt
+dataset: amass
+debug: false
+device: 3
+expname: exps
+folder: ./exps/9_15_feat_mld_vae_l1loss
+glob: true
+glob_rot:
+- 3.141592653589793
+- 0
+- 0
+jointstype: vertices
+lambda_rc: 100.0
+lambda_rcxyz: 100.0
+lambda_vel: 1.0
+lambda_velxyz: 1.0
+lambdas:
+  rc: 100.0
+  rcxyz: 100.0
+latent_dim: 512
+losses:
+- rc
+- rcxyz
+lr: 0.0001
+max_len: -1
+min_len: -1
+modelname: motionclip_transformer_rc_rcxyz
+modeltype: motionclip
+num_epochs: 500
+num_frames: 60
+num_layers: 9
+num_seq_max: -1
+only_60_classes: false
+pose_rep: feats
+sampling: conseq
+sampling_step: 1
+snapshot: 20
+translation: true
+use_action_cat_as_text_labels: false
+use_only_15_classes: false
+vertstrans: false
+nfeats: 1
+njoints: 263
+num_classes: 1

configs/modules/scheduler.yaml

@@ -0,0 +1,36 @@
+# scheduler:
+#   target: diffusers.DDIMScheduler
+#   num_inference_timesteps: 50
+#   eta: 0.0
+#   params:
+#     num_train_timesteps: 1000
+#     beta_start: 0.00085
+#     beta_end: 0.012
+#     beta_schedule: 'scaled_linear' # Optional: ['linear', 'scaled_linear', 'squaredcos_cap_v2']
+#     clip_sample: false # clip sample to -1~1
+#     set_alpha_to_one: false
+#     steps_offset: 1
+#     prediction_type: 'epsilon' # Optional: ['epsilon', 'sample']
+
+# noise_scheduler:
+#   target: diffusers.DDPMScheduler
+#   params:
+#     num_train_timesteps: 1000
+#     beta_start: 0.00085
+#     beta_end: 0.012
+#     beta_schedule: 'scaled_linear' # Optional: ['linear', 'scaled_linear', 'squaredcos_cap_v2']
+#     variance_type: 'fixed_small'
+#     clip_sample: false # clip sample to -1~1
+#     prediction_type: 'epsilon' # Optional: ['epsilon', 'sample']
+
+scheduler:
+  target: diffusers.FlowMatchEulerDiscreteScheduler
+  num_eval_steps: 51
+  num_demo_steps: 201
+  params:
+    num_train_timesteps: 1000
+
+noise_scheduler:
+  target: diffusers.FlowMatchEulerDiscreteScheduler
+  params:
+    num_train_timesteps: 1000

configs/modules/text_encoder.yaml

@@ -0,0 +1,7 @@
+text_encoder:
+  target: hftrainer.models.motion.motionlab.network.rfmotion.models.architectures.rfmotion_clip.RFMotionTextEncoder
+  params:
+    finetune: false 
+    last_hidden_state: true
+    modelpath: ${model.clip_path}
+    max_length: 77

mean_motion.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6574331d90d01bd3969df88fdae2de5f475ec3b6328b214a398c60bc686e65c7
+size 392

model_index.json

@@ -0,0 +1,19 @@
+{
+  "_class_name": "MotionLabPipeline",
+  "_library_name": "hftrainer",
+  "model_type": "motionlab",
+  "format": "hftrainer-motionlab-artifact-v1",
+  "bundle_class": "hftrainer.models.motion.motionlab.bundle.MotionLabBundle",
+  "pipeline_class": "hftrainer.pipelines.motionlab.pipeline.MotionLabPipeline",
+  "artifacts": {
+    "checkpoint": "motionflow.ckpt",
+    "configs": "configs",
+    "mean": "Mean.npy",
+    "std": "Std.npy",
+    "mean_motion": "mean_motion.npy",
+    "std_motion": "std_motion.npy"
+  },
+  "api": {
+    "from_pretrained": "hftrainer.pipelines.motionlab.MotionLabPipeline.from_pretrained"
+  }
+}

motionflow.ckpt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4137a5d1b488329f603aff89bb677336ab4092d578b15c007a1ea7acfb23657f
+size 2931450169

std_motion.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0971c47de2a1946e8c6bf1a83ad46e74d8aac89372a6d847fad573cfe8f8201f
+size 392