OzzyGT/Pixal3D_sdnq_dynamic_4bit

Pixal3D SDNQ INT4

bf16	INT4

Left: original bf16 · Right: this INT4 model (same input, camera and seed).

This is a dynamic 4-bit quantized version of OzzyGT/Pixal3D, the Diffusers conversion of TencentARC/Pixal3D, using SDNQ. The four flow transformers (structure, shape 512, shape 1024 and texture) are quantized; the DINOv3 image encoder, the NAF feature upsampler and the decoders are unchanged. The repository is 5.4 GB instead of 13 GB.

Like other quantized diffusion models, the same seed gives a slightly different result than bf16, more so at lower bit-widths.

Note: You need SDNQ v0.2.8 or superior.

Usage

Install PyTorch for your GPU, then:

pip install diffusers==0.41.0 transformers==5.20.2 "accelerate>=1.15" "safetensors>=0.8" numpy pillow "sdnq>=0.2.8"

Pixal3D runs with the Pixal3D Modular blocks, with components_repo pointing at this repository. Import sdnq before loading so the quantized weights are registered. Fully on the GPU, the sample below peaked at about 24 GiB of allocated VRAM; most of that is activations of the sparse decoding stages, not weights.

The example uses trust_remote_code=True and trust_components_code=True.

Note: both flags run Python code downloaded from the Hub: trust_remote_code=True runs the code of the blocks repository, and trust_components_code=True runs the code shipped in the components_repo. Only use them with repositories you trust, and review their code before passing different ones.

Sample

The INT4 video above was generated with the following code (seed 42, 12 steps, 1024 resolution):

import math

import sdnq  # noqa: F401
import torch
from diffusers import ModularPipelineBlocks
from diffusers.utils import load_image


blocks = ModularPipelineBlocks.from_pretrained(
    "OzzyGT/Pixal3D_Modular",
    trust_remote_code=True,
    components_repo="OzzyGT/Pixal3D_sdnq_dynamic_4bit",
    trust_components_code=True,
)
pipe = blocks.init_pipeline()
flow_names = (
    "sparse_structure_flow_model",
    "shape_slat_flow_model_512",
    "shape_slat_flow_model_1024",
    "tex_slat_flow_model_1024",
)
dtypes = {spec.name: torch.float32 for spec in blocks.expected_components}
dtypes.update({name: torch.bfloat16 for name in (*flow_names, "image_encoder")})
pipe.load_components(dtype=dtypes)
pipe.to("cuda")

image = load_image("https://huggingface.co/datasets/OzzyGT/diffusers-examples/resolve/main/Pixal3D/Pixal3D_jester_input.png")
result = pipe(
    image=image,
    camera_angle_x=math.radians(33.85),
    distance=1.643,
    mesh_scale=1.0,
    resolution=1024,
    num_inference_steps=12,
    generator=torch.Generator().manual_seed(42),
    output=["geometry", "materials", "actual_resolution"],
)
geometry = result["geometry"]
materials = result["materials"]

The input is a prepared image (subject on black, no margin); the camera (horizontal FOV 33.85°, distance 1.643) was estimated with MoGe-2. The pipeline returns flexible-dual-grid geometry and voxel materials, not a mesh; the video shows a mesh extracted from them with vertex colors.

Licenses

• Pixal3D weights and model code: MIT, from TencentARC/Pixal3D. Code adapted from Pixal3D and TRELLIS.2 keeps the Tencent and Microsoft attribution; see NOTICE.
• Image encoder: DINOv3 License, from facebook/dinov3-vitl16-pretrain-lvd1689m. Built with DINOv3.
• Feature upsampler: Apache-2.0, from valeoai/NAF. Its RoPE code keeps Meta's DINOv3 terms.