README.md

metadata

license: apache-2.0
tags:
  - motion-generation
  - human-pose
  - music-video
  - transformer
  - boostify
  - motiondna
library_name: pytorch
pipeline_tag: other

Boostify MotionDNA v0

MotionDNA is Boostify's human-motion model: a compact autoregressive Transformer that learns the DNA of how performers move in real music videos and can generate new, plausible motion sequences (2D skeletal pose) from a short seed.

This is the v0 checkpoint, trained end-to-end on motion extracted from real music-video clips. It is intentionally lightweight so it runs (and trains) on a single NVIDIA L4 (23 GB) — in fact training uses under 1 GB of VRAM.

What it does

• Input: a sequence of 2D human poses (17 COCO keypoints, x/y).
• Output: the predicted next pose, applied autoregressively to roll out new motion.
• Use cases: motion previews, choreography ideation, b-roll motion guides, music-video editing assistance inside Boostify Dataset Studio.

Architecture

• Causal Transformer encoder (next-pose prediction).
• d_model=256, n_head=8, n_layer=6, window WIN=64, input dim 34 (17×2).
• ~4.77M parameters. Smooth-L1 loss. AdamW + cosine schedule.

Training data

Pose sequences extracted with YOLOv8-pose from real music-video clips (46 clips, 11,038 frames). Poses are hip-centered and torso-scaled, then standardized (mean/std stored inside the checkpoint).

Source footage is used internally for research/demo. The released artifact contains only the trained weights and normalization stats — no source video.

Files

File	Description
motiondna_v0_best.pt	Best checkpoint (lowest train loss).
motiondna_v0_final.pt	Final-epoch checkpoint.
motiondna_sample.npy	Example generated motion [136, 17, 2].
loss_history.json	Per-epoch training loss.
extract_poses.py	Pose-extraction pipeline (YOLOv8-pose → .npy).
train_motiondna.py	Training script.

Inference

import torch, numpy as np
ckpt = torch.load("motiondna_v0_best.pt", map_location="cpu")
cfg = ckpt["cfg"]; mean = ckpt["mean"]; std = ckpt["std"]
# rebuild the MotionDNA module from train_motiondna.py, load_state_dict(ckpt["model"]),
# seed with N normalized poses and roll out model(x)[:, -1:] autoregressively,
# then de-normalize with: pose = gen * std + mean

Limitations

• v0 trained on a small clip set → motion is coherent but limited in style range.
• 2D pose only (no 3D, no global translation, no audio conditioning yet).
• Roadmap: more clips, music/beat conditioning, 3D lift, style control.

Citation

Boostify MotionDNA (2026). Trained with the Boostify Dataset Studio pipeline.