| # Companion Forge v7 — Distilled 3D Model Stack |
|
|
| Target: **Hugging Face Jobs `l4x1` / NVIDIA L4 (SM89, 24 GB VRAM)**. |
|
|
| v7 keeps the validated v6.4 ONNX/TensorRT runtime as the production fallback and adds a training/optimization layer inspired by the Kimi-K3 work: progressive few-step distillation, selective FP8, 2:4 structured sparsity, DINOv3 conditioning, multi-view fusion, geometry teachers, symmetry guidance, geometry-aware MoE, and timestep caching. |
|
|
| ## Pipeline |
|
|
| ```text |
| image/text |
| -> DINOv3 bridge + optional multi-view fusion |
| -> SS flow student (10 -> 4 -> 2 -> 1 steps) |
| -> SLat flow student (10 -> 4 -> 2 -> 1 steps) |
| -> full custom ONNX/TensorRT SLat DAE |
| -> MeshFlow/TRELLIS teacher losses during training only |
| -> AniGen skeleton/skin teacher |
| -> symmetry + rig guidance |
| -> rigged GLB |
| ``` |
|
|
| ## What is implemented |
|
|
| - `training/distill_core.py`: progressive Euler macro-step KD + endpoint consistency + MeanFlow-style interval consistency. |
| - `training/distill_flow.py`: L4-friendly LoRA-then-merge training for SS/SLat flow students, stages `10→4→2→1`. |
| - `training/fp8_sparse.py`: selective ModelOpt FP8 hooks and exact magnitude-based 2:4 pruning for eligible Linear weights. |
| - `training/dinov3_bridge.py` + `distill_dinov3.py`: DINOv3 ViT-S bridge to AniGen's fixed `1374×1024` conditioning contract, distilled against DINOv2 ViT-L/14-reg. |
| - `runtime/multiview.py`: pose-Fourier multi-view token fusion for front/left/back/right conditioning. |
| - `runtime/symmetry.py`: flow-time reflection/C2/C4 velocity symmetrization plus rig symmetry loss. |
| - `training/teacher_hybrid.py`: cached MeshFlow/TRELLIS geometry teacher + AniGen rig teacher interface. |
| - `training/geometry_moe.py`: ModernMOE-inspired shared + top-k routed 3D experts. |
| - `runtime/timestep_cache.py`: residual timestep cache for >=4-step students; automatically unnecessary after 1–2-step distillation. |
| - `bench/quality_eval.py`: geometry/symmetry/joint/speed comparison harness. |
|
|
| ## L4 training order |
|
|
| 1. Distill DINOv3 bridge (keep current DINOv2 TRT runtime as fallback). |
| 2. Build teacher cache using v6.4 outputs plus optional MeshFlow/TRELLIS geometry targets. |
| 3. Distill SS `10→4`, SLat `10→4`; validate. |
| 4. Apply selective FP8 PTQ; if quality drops, run short QAT. |
| 5. Apply 2:4 pruning to MLP/projection weights, freeze masks, recovery distillation. |
| 6. Distill `4→2`, then `2→1` only if geometry/rig metrics stay within thresholds. |
| 7. Train geometry-inductive MoE as an optional higher-capacity student; distill it back to a dense/sparse deployment student if its routing overhead is not worthwhile on L4. |
| 8. Re-export ONNX external-data and compile SM89 TensorRT engines. Output heads/mesh remain FP32; sparse topology ops remain the validated custom plugins. |
|
|
| ## Safety/quality gates |
|
|
| A new student is not promoted merely because it is faster. Promotion requires finite tensors, exit code 0, GLB skin/skeleton validation, no lazy PyTorch fallback, and quality thresholds against the current v6.4 teacher. `1+1` is therefore an experimental final stage, not an automatic default. |
|
|
| ## External technology mapping |
|
|
| - NVIDIA FastGen concepts: progressive KD / MeanFlow / consistency-style few-step training. |
| - NVIDIA Model Optimizer: FP8 PTQ/QAT and export path. |
| - TensorRT: 2:4 structured sparse tactics where eligible. |
| - Meta DINOv3: stronger dense visual conditioning. |
| - Meta MeshFlow / Microsoft TRELLIS.2: geometry/topology teacher targets only; licensing/runtime isolation is preserved. |
| - SymTRELLIS idea: velocity symmetrization adapted to AniGen flow and rig constraints. |
| - ModernMOE idea: routed/shared lightweight experts specialized for geometry roles. |
|
|
| The current v6.4 runtime remains the rollback target until each v7 stage has its own L4 E2E validation artifact. |
|
|
