File size: 2,932 Bytes
d328c79 d67a91b d328c79 d67a91b d328c79 d67a91b d328c79 d67a91b d328c79 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 | # Companion Forge L4 ONNX/TensorRT Runtime
Precompiled ONNX and TensorRT runtime for `patdev/Companion-Forge`, targeted at Hugging Face Jobs `l4x1` (NVIDIA L4 / SM89 / 24 GB class VRAM).
## Production coverage
| Component | ONNX | TensorRT L4 | Notes |
|---|---|---|---|
| FLUX.2 Klein 4B transformer | opset 23 | ✅ | static 512 profile |
| DINOv2 ViT-L/14-reg | ✅ | ✅ | 4.83× vs ORT CUDA in validation |
| DSINE | ✅ | ✅ | 3.62× vs native PyTorch |
| AniGen `ss_flow_solo` | opset 23 | ✅ | complete model; 1.28× per forward |
| AniGen `ss_dae` decoder | opset 23 | ✅ | complete model; 2.02× |
| AniGen `slat_flow_auto` transformer core | opset 18 dynamic | ✅ | 7.30× vs sparse native core |
| AniGen SLat sparse shell | — | native spconv | only ~188 MB shell weights; core lives in TRT |
| AniGen skin decoder | opset 18 dynamic | ✅ | 6.80×; profile up to 300k vertices / 64 joints |
| AniGen SLat decoder sparse backbone | custom sparse ops | native spconv + FlashAttention | windowed sparse attention/topology changes are not standard dense ONNX ops |
| BiRefNet/rembg fallback | upstream ONNX | ORT GPU | used only for complex backgrounds |
The remaining native path is not another dense model: it is the sparse/topology operator layer (`SparseConv3D`, window-partitioned sparse attention, sparse up/downsample and subdivide). TensorRT Plugin V3 schemas are tracked under `plugins/` for future replacement.
## Final v6.4 L4 validation
Fast rigged Code-Pet generation completed end-to-end with:
- runtime: `hybrid-staged-dino-trt+dsine-trt+slat-shell-trt+skin-trt+ss-trt`
- worker time: **90.443 s** including cold model/runtime preparation
- conditioning: **13.939 s**
- runtime/model load: **12.997 s**
- SS + SLat sampling: **7.895 s**
- mesh post-process: **3.169 s**
- PyTorch-tracked peak: **6.972 GiB**
- real CUDA device peak: **12.734 GiB**
- output: rigged `companion.glb` + separate `skeleton.glb`
- no PyTorch SS/SLat flow-model fallback in the validated run
The earlier hybrid implementation took about **115.5 s** and loaded full native AniGen flow checkpoints. v6.4 removes those flow checkpoints from the normal path.
## Important artifacts
- `onnx/flux2-klein-4b/512/`
- `onnx/dinov2/`
- `onnx/dsine/`
- `onnx/anigen/ss-flow-solo/`
- `onnx/anigen/ss-decoder/`
- `onnx/anigen/slat-flow-core/`
- `onnx/anigen/skin-decoder/`
- `engines/l4-sm89/`
- `weights/anigen/slat-flow-shell/`
- `runtime/`
- `bench/`
- `plugins/`
## SLat split
The original `slat_flow_auto` checkpoint is about **2.46 GB**. The production split keeps only the learned sparse IO shell (~188 MB) and moves the 24 heavy transformer blocks into the SM89 TensorRT plan (~1.71 GB). The full PyTorch checkpoint is downloaded only by the lazy fallback when an input exceeds the validated dynamic engine profile.
See `runtime_manifest.json` for the machine-readable runtime order, profiles and validation records.
|