| --- |
| license: apache-2.0 |
| base_model: speridlabs/idol |
| pipeline_tag: image-to-3d |
| tags: |
| - diffusers-3d |
| - idol |
| - 3d |
| - image-to-3d |
| - gaussian-splatting |
| --- |
| |
| # IDOL for diffusers-3d |
|
|
| [speridlabs/idol](https://huggingface.co/speridlabs/idol) converted into a [diffusers-3d](https://github.com/suvadityamuk/diffusers-3d) |
| pipeline: ordinary Diffusers and Transformers component folders that the installed package loads. Nothing here |
| requires remote code. |
|
|
| ## Install |
|
|
| ```bash |
| pip install git+https://github.com/suvadityamuk/diffusers-3d.git |
| pip install "git+https://github.com/suvadityamuk/diffusers-3d.git#subdirectory=packages/diffusers-3d" |
| ``` |
|
|
| `diffusers-3d` runs every network in plain PyTorch on CPU or GPU. Rendering Gaussian splats needs the optional |
| `gsplat` backend; the PBR-textured mesh for TRELLIS.2 needs CuMesh (see the package docs). |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from diffusers_3d import AutoPipelineForImageTo3D, ImageCondition |
| |
| pipeline = AutoPipelineForImageTo3D.from_pretrained("suvadityamuk/idol-diffusers-3d").to("cuda") |
| splat = pipeline(ImageCondition(image=rgba), mode="real", generator=torch.Generator("cuda").manual_seed(42)).objects[0] |
| camera = splat.metadata["camera"] # predicted yaw, pitch, radius, and FOV of the input view |
| splat.save("object.ply") |
| ``` |
|
|
| **DINOv3 and the FLUX.2 VAE are not included.** Both are gated third-party weights, so |
| `from_pretrained` loads `image_encoder` from |
| [facebook/dinov3-vith16plus-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vith16plus-pretrain-lvd1689m) |
| (DINOv3 License) and `vae` from the `vae/` folder of |
| [black-forest-labs/FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev) (FLUX non-commercial license), |
| each at a pinned revision. Accept both licenses and log in (`hf auth login`) first; without access, loading raises an |
| error that says so. Inputs without usable alpha are cut out with BiRefNet using |
| [ZhengPeng7/BiRefNet](https://huggingface.co/ZhengPeng7/BiRefNet) weights (MIT), as upstream does |
| (`pipeline.disable_background_removal()` skips it for inputs that already carry alpha). |
|
|
| `mode="synthetic"` (the default, for renders) crops 1.2 times the object; `mode="real"` (photos) crops 1.7 times. Defaults match upstream: 50 steps, shift 3.0, 262144 Gaussians, `guidance_scale=4.0` (upstream's `cfg_strength=3.0` in the Diffusers convention). Without `dtype`, each component loads in upstream's precision; pass `dtype=torch.float32` on CPU. |
|
|
| ## Components |
|
|
| | Folder | Class | |
| |---|---| |
| | `decoder` | `diffusers_3d.families.triposplat.decoder.TripoSplatGaussianDecoder` | |
| | `image_encoder` | `transformers.DINOv3ViTModel` (not included; loads from `facebook/dinov3-vith16plus-pretrain-lvd1689m`) | |
| | `scheduler` | `diffusers.FlowMatchEulerDiscreteScheduler` | |
| | `transformer` | `diffusers_3d.families.triposplat.models.TripoSplatTransformerModel` | |
| | `vae` | `diffusers_3d.families.triposplat.models.TripoSplatImageVAE` (not included; loads from `black-forest-labs/FLUX.2-dev`) | |
|
|
| ## Provenance |
|
|
| Converted with `diffusers-3d-convert-idol` from `diffusers-3d 0.1.0.dev0`; the package's family README lists the pinned upstream |
| revisions and how the conversion maps each tensor. Parity tests against the pinned upstream code are part of the |
| package test suite. |
|
|
| ## License and attribution |
|
|
| IDOL weights and code: Apache-2.0, Copyright (c) Speridlabs. IDOL keeps the architecture of |
| VAST-AI-Research TripoSplat (MIT, `LICENSE-MIT`). The DINOv3 and FLUX.2 VAE encoders are not redistributed; they |
| load from their original repositories under their own licenses. See `LICENSE-APACHE-2.0` and `NOTICE`. Not |
| affiliated with or endorsed by Speridlabs, VAST, Meta, or Black Forest Labs. |
|
|
