| --- |
| license: cc-by-4.0 |
| tags: |
| - animation |
| - motion-in-betweening |
| - onnx |
| - 3d |
| - qtmesheditor |
| library_name: onnx |
| pipeline_tag: other |
| --- |
| |
| # QtMeshEditor — RMIB Animation In-Betweening |
|
|
| A small transformer that fills the gap between two sparse keyframes with smooth, |
| plausible intermediate motion — **Robust Motion In-Betweening (RMIB)** style, |
| exported to **ONNX** for in-app inference via ONNX Runtime. |
|
|
| Built for **[QtMeshEditor](https://github.com/fernandotonon/QtMeshEditor)** (issue #409) — a free, |
| open-source 3D mesh & animation editor — and its companion **QtMesh Cloud** |
| asset service. The app downloads this model on first use and runs it locally |
| (offline) to power the dope-sheet **"AI in-between → Fill gap"** action, the |
| `qtmesh anim --in-between` CLI, and the `motion_in_between` MCP tool. |
|
|
| ## Model |
|
|
| - **Input:** `pose_pair` `float32 [1, 2, 220]` — the start and end pose, each |
| 22 core-body joints × 10 DoF `[tx,ty,tz, qx,qy,qz,qw, sx,sy,sz]`. |
| - **Output:** `interior` `float32 [1, 8, 220]` — 8 predicted in-between poses. |
| - **Architecture:** a 4-layer Transformer encoder (start + end pose tokens + |
| learnable interior-frame queries → per-frame pose). ~13 MB. |
| - **Skeleton:** the 22 CMU core-body joints (hips / spine / neck / head + both |
| arms + both legs). QtMeshEditor maps arbitrary rig bone names (Mixamo, |
| generic, CMU) onto these roles at runtime; rigs that don't map fall back to a |
| deterministic spline. |
|
|
| ## Training data & license |
|
|
| Trained **from scratch** on the **[CMU Graphics Lab Motion Capture Database](http://mocap.cs.cmu.edu)** |
| — permissively licensed (free to use/modify/redistribute, including in |
| commercial products; you may not resell the motion data itself; credit |
| mocap.cs.cmu.edu). This is what makes the weights redistributable under |
| QtMeshEditor's permissive (MIT) distribution — unlike the field-standard LAFAN1 |
| (CC-BY-NC-ND), which we deliberately avoided. |
|
|
| On a held-out CMU split the model beats spherical-linear interpolation (the |
| shipped spline fallback) by **more than 2×** on rotation error. |
|
|
| Weights released under **CC-BY-4.0**; please credit *QtMeshEditor* and the CMU |
| Graphics Lab Motion Capture Database. |
|
|
| ## Reproducing |
|
|
| `scripts/export-rmib-onnx.py` in the QtMeshEditor repo (one-time, offline; not |
| shipped with the app). |
|
|
| ## Versions |
|
|
| - **v1.0.0** (current) — initial RMIB in-betweening model trained on CMU MoCap |
| (22 core-body joints, ~421k windows; beats slerp by >2× on a held-out split). |
|
|
