| --- |
| license: apache-2.0 |
| tags: [onnx, mediapipe, blendshapes, arkit, face-capture, qtmesheditor] |
| --- |
| |
| # Face blendshapes MLP-Mixer (ONNX) |
|
|
| Google MediaPipe's face blendshape model (MLP-Mixer), converted to ONNX. |
|
|
| - **Input** `[1,146,2]` — the 146-landmark subset as **pixel coords** |
| (`x·imgW, y·imgH`). |
| - **Output** `[52]` — **52 ARKit-compatible blendshape scores** in `[0,1]`, |
| in MediaPipe order (`_neutral`, `browDownLeft`, … `jawOpen`, … |
| `mouthSmileLeft/Right`, … `eyeBlinkLeft/Right`, …). These drive ARKit-style |
| morph targets directly. |
|
|
| Converted with tf2onnx optimizers disabled (the optimized graph broke a |
| LayerNorm under ONNX Runtime — see the spike doc). |
|
|
| ## License |
|
|
| Apache-2.0 — this graph is a direct ONNX conversion of a Google |
| [MediaPipe](https://developers.google.com/mediapipe) model (Apache-2.0 code |
| AND weights). Conversion + numerical-parity proof (vs the Python `mediapipe` |
| reference): [`scripts/export-facecap-onnx.py`](https://github.com/fernandotonon/QtMeshEditor/blob/master/scripts/export-facecap-onnx.py), |
| contract in [`docs/MOCAP_SPIKE.md`](https://github.com/fernandotonon/QtMeshEditor/blob/master/docs/MOCAP_SPIKE.md). |
|
|
| ## How it is used |
|
|
| Mirror of one graph from [`fernandotonon/QtMeshEditor-models`](https://huggingface.co/fernandotonon/QtMeshEditor-models) |
| (`mocap/…`), which [QtMeshEditor](https://github.com/fernandotonon/QtMeshEditor) |
| downloads on first use for its **Performance Capture** feature (video/webcam → |
| facial morph + head + full-body skeletal animation, epic #869). This standalone |
| repo is for discoverability; the app fetches from the aggregate repo. |
|
|
