README.md

metadata

title: Companion Forge
emoji: 🤖
colorFrom: indigo
colorTo: blue
sdk: gradio
sdk_version: 6.2.0
app_file: app.py
pinned: false
license: mit
hf_oauth: true
hf_oauth_expiration_minutes: 480
hf_oauth_scopes:
  - jobs
  - contribute-repos
models:
  - black-forest-labs/FLUX.2-klein-4B
  - VAST-AI/AniGen
tags:
  - text-to-3d
  - image-to-3d
  - rigging
  - animation
  - companion
  - game-assets
  - hf-jobs
  - onnx
  - tensorrt
  - distillation

Companion Forge v7 Lab — v6.4 production + next-generation L4 training

Create a rigged desktop Code Pet from text or an image. The Gradio UI stays on CPU; generation and v7 training jobs are dispatched on demand to Hugging Face Jobs l4x1 (NVIDIA L4 / SM89, 8 vCPU, ~30 GB RAM). GPU jobs terminate after the requested work.

Stable production generation

The public generation path remains the validated v6.4 runtime until a v7 student passes quality, finite-tensor and rigging gates:

FLUX.2 Klein TRT → DINOv2 TRT + DSINE TRT → AniGen SS TRT → AniGen SLat TRT → full custom sparse/topology TensorRT decoder → Skin TRT → rigged GLB + skeleton + behavior bundle.

The SLat decoder no longer depends on an ordinary native PyTorch backbone in the normal path. Companion Forge has custom ONNX/TensorRT Plugin V3 operators for:

• SparseConv3D
• SparseWindowAttention
• SparseDownsample
• SparseUpsample
• SparseSubdivide
• MeshTopologyExtract
• SparseMeshTopologyExtract

The learned decoder branches/heads, topology extraction and skinning are validated end to end on L4; native checkpoints are lazy fallbacks only.

V7 Lab

The V7 Lab tab exposes an experimental training/optimization stack without replacing stable inference automatically.

Few-step flow distillation

Both major AniGen flows have a real L4 training path using one shared teacher/student base plus compact LoRA deltas:

• SS Flow: progressive 10 → 4 → 2 → 1
• SLat Flow: progressive 10 → 4 → 2 → 1
• velocity KD + endpoint consistency
• MeanFlow-style interval consistency
• later stages merge all previous adapters in order before learning the next delta

Real one-step backward smokes already pass on L4 for both SS and sparse SLat. Stage adapters are ~30 MB rather than multi-gigabyte duplicate checkpoints.

FP8 + 2:4

The v7 optimization pipeline includes:

• NVIDIA Model Optimizer FP8 calibration/QAT on a CUDA devel Job image;
• real calibration tensors captured from private teacher-cache records;
• sensitive geometry/output heads, mesh/topology and normalization paths kept out of aggressive quantization;
• 2:4 pruning on eligible MLP/projection weights;
• TensorRT SPARSE_WEIGHTS tactics, with TensorRT free to keep a dense tactic when it benchmarks faster for a specific shape;
• ONNX external-data export and precompiled SM89 TensorRT plans.

DINOv3, multi-view, symmetry, MoE and caching

Implemented v7 modules include:

• a DINOv3 → exact AniGen 1374 × 1024 conditioning bridge and DINOv2 feature-distillation trainer;
• front/left/back/right multi-view fusion with pose Fourier encoding, plus a shape-preserving mean-fusion fallback;
• flow-time reflection/C2/C4 velocity symmetrization and rig bilateral priors;
• Geometry-Inductive MoE experts for coarse_shape, surface_detail, limbs, head_face, skeleton, skin, and materials;
• time-aware residual/timestep caching for students with ≥4 sampling steps; caching is bypassed after successful 1–2 step distillation.

The official Meta DINOv3 weights currently require gated Hub access for the signed-in account, so the v7 bridge is ready but production continues to use DINOv2 TensorRT until access is available. MeshFlow is treated as an optional research geometry teacher and is not silently enabled in production.

Private v7 training data

Companion Forge does not automatically collect generated references for training. A reference is added only when the signed-in user explicitly clicks Add to private v7 cache.

Each user gets app-created private repositories for:

• companion-forge-v7-teacher-cache
• companion-forge-v7-students

The lightweight teacher record stores conditioning plus sparse coordinates; random latent noise is regenerated during training. This avoids storing large trajectories and improves training diversity.

Promotion policy

A v7 student is not promoted because it is merely faster. Promotion requires:

1. finite tensors through the full pipeline;
2. no unexpected PyTorch fallback;
3. valid mesh, skeleton and skinning;
4. geometry/rig metrics against the v6.4 teacher;
5. an L4 end-to-end benchmark;
6. successful ONNX/TensorRT export and runtime validation.

Until then, the v6.4 generation runtime remains the rollback-safe production path.