---
license: cc0-1.0
tags:
  - text-to-motion
  - skeletal-animation
  - qtmesheditor
  - experimental
---

# QtMeshEditor Text-to-Motion (experimental, #411)

A small, **experimental** from-scratch text-to-motion model for
[QtMeshEditor](https://github.com/fernandotonon/QtMeshEditor). Given a text
prompt (an action keyword), it generates a 60-frame @30fps, 22-joint
**canonical WORLD-frame** skeletal clip that QtMeshEditor retargets onto an
arbitrary humanoid rig.

> The model QtMeshEditor actually downloads at runtime lives in the shared
> [`fernandotonon/QtMeshEditor-models`](https://huggingface.co/fernandotonon/QtMeshEditor-models)
> repo under `motion/`. This repo is the standalone model card + mirror.

## Status: experimental

The shipped **default** in QtMeshEditor is the deterministic **template-clip
retarget** (a curated library of 47 real CMU mocap clips across 15 actions,
with per-action variety) — that is the quality bar. This model is an opt-in
(`--model` / GUI checkbox / MCP `model:true`) that **falls back to the
template** automatically when unavailable or out of vocabulary. It produces
coherent, upright motion with per-generate variety, but is stylistically
gentler/less crisp than the real-mocap templates.

## Training data — permissive only

Trained from scratch on **clean, dynamic, single-action windows** mined from
the **CMU MoCap** database (commercial-OK). AMASS / HumanML3D / KIT-ML were
**excluded** (non-commercial). Windows are 30fps, 2s, selected for motion
energy and snapped to a calm near-neutral start frame; mirror-augmented.

## Architecture (v4)

- **6D-rotation** representation (Zhou et al. 2019), correctly column-packed.
- Cross-attention transformer decoder with an **absolute** per-frame pose head
  (self-attention models temporal coherence; no error-accumulating cumsum).
- CVAE latent with z=0 supervision + aggregate-posterior matching.
- Per-sample velocity/acceleration matching in both 6D and true rotation
  (geodesic) space; derived-local supervision (the quantity the retarget
  renders); 1-2-1 output smoothing baked into the ONNX graph.
- ~7.6M params, exports to ONNX (one forward pass).

## I/O contract

```
input  "tokens" float32 [1, V]   one-hot over the fixed action vocab (see t2m-vocab.json)
input  "seed"   float32 [1, Z]   latent noise (host samples ~N(0,0.5) and does best-of-N)
output "motion" float32 [1, T, C]  C = 22*10 per-joint [tx,ty,tz, qx,qy,qz,qw, sx,sy,sz]
```

`t2m-vocab.json` ships the `{vocab, Z, T, C, J, joints, fps, frame}` the host
needs — `frame: "world"` marks the WORLD-frame convention (retarget takes a
world delta), `fps: 30`. Vocabulary: walk, run, jump, dance, march, kick,
punch, wave, climb, sit, throw, boxing, idle.

## Reproducing

`scripts/prep-t2m-v4.py` + `scripts/train-t2m-onnx-v4.py` in the QtMeshEditor
repo (one-time, offline dev tools — the app never runs Python).