Pixal3D

Generated with the code below (seed 42, 12 steps, 1024 resolution).

A Diffusers conversion of TencentARC/Pixal3D for single-image 3D generation, including its DINOv3 image encoder. Built with DINOv3.

For model details, see the original model card and Pixal3D repository.

Usage

Install PyTorch for your GPU, then:

pip install diffusers==0.41.0 transformers==5.20.2 "accelerate>=1.15" "safetensors>=0.8" numpy pillow

The model runs with the Pixal3D Modular blocks, with components_repo pointing at this repository.

The example uses trust_remote_code=True and trust_components_code=True.

Note: both flags run Python code downloaded from the Hub: trust_remote_code=True runs the code of the blocks repository, and trust_components_code=True runs the code shipped in the components_repo. Only use them with repositories you trust, and review their code before passing different ones.

import math

import torch
from diffusers import ModularPipelineBlocks
from diffusers.utils import load_image


blocks = ModularPipelineBlocks.from_pretrained(
    "OzzyGT/Pixal3D_Modular",
    trust_remote_code=True,
    components_repo="OzzyGT/Pixal3D",
    trust_components_code=True,
)
pipe = blocks.init_pipeline()
flow_names = (
    "sparse_structure_flow_model",
    "shape_slat_flow_model_512",
    "shape_slat_flow_model_1024",
    "tex_slat_flow_model_1024",
)
dtypes = {spec.name: torch.float32 for spec in blocks.expected_components}
dtypes.update({name: torch.bfloat16 for name in (*flow_names, "image_encoder")})
pipe.load_components(dtype=dtypes)
pipe.to("cuda")

image = load_image("https://huggingface.co/datasets/OzzyGT/diffusers-examples/resolve/main/Pixal3D/Pixal3D_jester_input.png")
result = pipe(
    image=image,
    camera_angle_x=math.radians(33.85),
    distance=1.643,
    mesh_scale=1.0,
    resolution=1024,
    num_inference_steps=12,
    generator=torch.Generator().manual_seed(42),
    output=["geometry", "materials", "actual_resolution"],
)
geometry = result["geometry"]
materials = result["materials"]

The input is a prepared image (subject on black, no margin); the camera (horizontal FOV 33.85°, distance 1.643) was estimated with MoGe-2. The pipeline returns flexible-dual-grid geometry and voxel materials, not a mesh; the video shows a mesh extracted from them with vertex colors.

Licenses

• Pixal3D weights and model code: MIT, from TencentARC/Pixal3D. Code adapted from Pixal3D and TRELLIS.2 keeps the Tencent and Microsoft attribution; see NOTICE.
• Image encoder: DINOv3 License, from facebook/dinov3-vitl16-pretrain-lvd1689m. Built with DINOv3.
• Feature upsampler: Apache-2.0, from valeoai/NAF. Its RoPE code keeps Meta's DINOv3 terms.