Experimental from-scratch text-to-motion model (#411): CMU+Quaternius CC0, ONNX

README.md

@@ -0,0 +1,53 @@
+---
+license: cc0-1.0
+tags:
+  - text-to-motion
+  - skeletal-animation
+  - qtmesheditor
+  - experimental
+---
+
+# QtMeshEditor Text-to-Motion (experimental, #411)
+
+A small, **experimental** from-scratch text-to-motion model for
+[QtMeshEditor](https://github.com/fernandotonon/QtMeshEditor). Given a text
+prompt (an action keyword), it generates a 40-frame, 22-joint **canonical**
+skeletal clip that QtMeshEditor retargets onto an arbitrary humanoid rig.
+
+## Status: experimental
+
+Render-verified quality is **action-dependent**: locomotion (e.g. `walk`) is
+coherent; gestures (`wave`, `run`) can drift in the last few frames. The shipped
+default in QtMeshEditor is the deterministic **template-clip retarget**; this
+model is an opt-in (`--model` / GUI checkbox / MCP `model:true`) that **falls
+back to the template** automatically when unavailable or out of vocabulary.
+
+## Training data — permissive only
+
+Trained from scratch on **clean, dynamic, single-action windows** mined from:
+- **CMU MoCap** (commercial-OK) — the bulk.
+- **Quaternius Universal Animation Library** (CC0) — supplementary.
+
+AMASS / HumanML3D / KIT-ML were **excluded** (non-commercial). Idle/near-static
+and multi-action-labelled windows were filtered out (they dominated the raw set
+and caused pose collapse).
+
+## Architecture
+
+6D-rotation representation, cross-attention decoder with a delta-integration
+(cumsum) head for temporal continuity, CVAE latent, balanced sampling. ~7.6M
+params. Exports to ONNX (one forward pass).
+
+## I/O contract
+
+```
+input  "tokens" float32 [1, V]   one-hot over the fixed action vocab (see t2m-vocab.json)
+input  "seed"   float32 [1, Z]   latent (zeros = mean clip)
+output "motion" float32 [1, T, C]  C = 22*10 per-joint [tx,ty,tz, qx,qy,qz,qw, sx,sy,sz]
+```
+
+`t2m-vocab.json` ships the `{vocab, Z, T, C, J, joints}` the host needs to build
+the input and interpret the output.
+
+Vocabulary: walk, run, jog, jump, dance, march, climb, kick, punch, sit,
+stretch, throw, wave, boxing, turn, forward.

t2m-vocab.json

@@ -0,0 +1 @@
+{"vocab": ["walk", "run", "jog", "jump", "dance", "march", "climb", "kick", "punch", "sit", "stretch", "throw", "wave", "boxing", "turn", "forward"], "Z": 24, "T": 40, "C": 220, "J": 22, "joints": ["hip", "abdomen", "chest", "neck", "neck1", "head", "rcollar", "rshoulder", "relbow", "rhand", "lcollar", "lshoulder", "lelbow", "lhand", "rbuttock", "rhip", "rknee", "rfoot", "lbuttock", "lhip", "lknee", "lfoot"]}

t2m.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41da066e3ce127a4985a1b2aff2ccc0987153ff51904d1f5422cf35c16b9d704
+size 17569325