fernandotonon/QtMeshEditor-skintokens-onnx

SkinTokens / TokenRig — ONNX export

ONNX re-export of VAST-AI SkinTokens / TokenRig (MIT code + MIT weights, Qwen3-0.6B backbone, trained on Articulation-XL2.0 / CC-BY-4.0) — autoregressive ML skin-weight prediction: given a mesh and a skeleton, it predicts per-vertex bone weights. All credit for the original weights goes to VAST-AI-Research.

Exported for QtMeshEditor (issue #819), where it is the default skinner (qtmesh skin, the GUI "Compute Skin Weights" dialog, the rig→skin chain, and the compute_skin_weights MCP tool), running locally via ONNX Runtime with a geodesic-voxel fallback.

The files QtMeshEditor downloads at runtime live in the shared fernandotonon/QtMeshEditor-models repo under skintokens/. This repo is the standalone model card + mirror for people who want the converted weights themselves.

Files (five graphs + manifest)

file	role
mesh_cond.onnx	Michelangelo point-cloud encoder → LLM mesh-conditioning prefix
vae_cond.onnx	skin-CVAE conditioning encoder over the sampled points
embed.onnx	token id → LLM embedding
decoder.onnx + decoder.onnx.data	Qwen3-0.6B causal-LM KV-cache step (external weights — ONNX Runtime 1.20.1 segfaults parsing a >1.6 GB single-file proto)
skin_decode.onnx	FSQ skin tokens → per-joint, per-sampled-point weights (FSQ folded in)
skintokens.json	manifest: every config value the host needs (below)

Inference contract

1. Surface-sample num_points (8192) points + normals; normalise mesh+joints per upstream AugmentAffine (joints included in the AABB, exact [-1,1] fit).
2. Tokenize the skeleton teacher-forced (DFS order; per bone [branch?, parent-joint xyz, joint xyz] discretised to 256 bins over [-1,1]; bos=257, cls "articulation"=266, stream ends with the switch token eos=258). Multi-root topologies must be re-parented to the first root + DFS-reordered first.
3. Prefix = mesh-cond embeddings + skeleton token embeddings → autoregressive greedy decode of J × tokens_per_skin (4) skin tokens, constrained to the FSQ range [267, 33035); global EOS 33035; full vocab 33036.
4. Per joint: skin_decode on its 4 FSQ ids (− 267 offset) → weights over the 8192 sampled points; transfer to full-res vertices by 8-NN inverse-distance.

LLM dims (manifest): hidden 896, 28 layers, 8 KV heads, head_dim 128; tokens_skin_cond=384, CVAE latent 512, FSQ codebook 32768.

Raw predictions are deliberately diffuse — the upstream demo voxel-masks them by default. QtMeshEditor applies a geodesic-localisation pass (filter to geodesically-local bone sets + renormalise); consumers are advised to do the same (bleed 0.74 → 0.05 in our measurements).

Reproducing

scripts/export-skintokens-onnx.py in the QtMeshEditor repo (one-time, offline; bf16→fp32, forced eager attention, decomposed RMSNorm for opset 18, trace-friendly FPS). Parity vs PyTorch ≈ 1e-5 on every graph.

License

MIT (same as the upstream code and weights). Training data: Articulation-XL2.0 (CC-BY-4.0) — credit VAST-AI-Research.