| --- |
| language: |
| - en |
| license: other |
| license_name: idol-with-third-party-components |
| license_link: https://huggingface.co/OzzyGT/idol_diffusers/blob/main/NOTICE |
| library_name: diffusers |
| pipeline_tag: image-to-3d |
| base_model: |
| - speridlabs/idol |
| - facebook/dinov3-vith16plus-pretrain-lvd1689m |
| - black-forest-labs/FLUX.2-klein-4B |
| tags: |
| - idol |
| - modular-diffusers |
| - image-to-3d |
| - gaussian-splatting |
| --- |
| |
| # Idol Diffusers |
|
|
| <video controls loop muted playsinline src="https://huggingface.co/datasets/OzzyGT/diffusers-examples/resolve/main/idol/idol_dragon_seed42_fp16.mp4"></video> |
|
|
| *Generated with the code below (seed `42`, 50 steps, 262,144 Gaussians).* |
|
|
| A Diffusers conversion of [speridlabs/idol](https://huggingface.co/speridlabs/idol) for single-image 3D Gaussian splat generation. This repository holds only weights and configs: the denoiser, the octree decoder, the Gaussian decoder and its two image encoders, DINOv3 and the FLUX.2 VAE. |
|
|
| For model details, see the [original model card](https://huggingface.co/speridlabs/idol) and the [Idol SDK repository](https://github.com/speridlabs/idol-sdk). |
|
|
| ## Usage |
|
|
| Install PyTorch for your GPU, then: |
|
|
| ```bash |
| pip install diffusers==0.42.1 transformers==5.14.1 "accelerate>=1.14" "safetensors>=0.8" numpy pillow |
| ``` |
|
|
| The model runs with the [Idol Modular Diffusers](https://huggingface.co/OzzyGT/idol_custom_blocks) blocks, which ship the model code and load the weights from this repository. |
|
|
| The example uses `trust_remote_code=True`. |
|
|
| > **Note:** `trust_remote_code=True` runs Python code downloaded from the Hub, here the code of the blocks repository. Only use it with repositories you trust, and review their code before passing a different one. |
|
|
| ```python |
| import torch |
| from diffusers import ModularPipeline |
| from diffusers.utils import load_image |
| |
| |
| pipe = ModularPipeline.from_pretrained("OzzyGT/idol_custom_blocks", trust_remote_code=True) |
| pipe.load_components(dtype=torch.bfloat16) |
| pipe.to("cuda") |
| |
| image = load_image("https://huggingface.co/datasets/OzzyGT/diffusers-examples/resolve/main/idol/nb-dragon-anime_input.png") |
| result = pipe( |
| image=image, |
| generator=torch.Generator("cuda").manual_seed(42), |
| output=["gaussians", "camera_latents"], |
| ) |
| gaussians = result["gaussians"] |
| camera_latents = result["camera_latents"] |
| ``` |
|
|
| To save the result as a standard 3DGS `.ply` file that any Gaussian splat viewer opens, use the [Idol recipe](https://github.com/asomoza/diffusers-recipes/blob/main/models/idol/scripts/base_example.py) from diffusers-recipes. |
|
|
| The input is a prepared image (1024x1024, the object cropped with a margin and composited on black), here prepared with Idol's own background removal and crop. The pipeline returns activated Gaussians (positions, colors, scales, rotations, opacities) and the predicted input camera, not a file; the video shows them rendered on a turntable. |
|
|
| Built with DINOv3. |
|
|
| ## Licenses |
|
|
| - Idol weights: [Apache-2.0](LICENSE), from [speridlabs/idol](https://huggingface.co/speridlabs/idol). See [NOTICE](NOTICE). |
| - Image encoder: [DINOv3 License](LICENSE_DINOv3), from [facebook/dinov3-vith16plus-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vith16plus-pretrain-lvd1689m), stored in bfloat16. |
| - FLUX.2 VAE: [Apache-2.0](LICENSE), from [black-forest-labs/FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B), unchanged. |
|
|
