| --- |
| license: mit |
| pipeline_tag: image-to-3d |
| tags: |
| - image-to-3d |
| - 3d-generation |
| - mesh-generation |
| - pbr |
| - pixal3d |
| --- |
| |
| # Pixal3D — image-to-3D with PBR materials (full precision) |
|
|
| High-fidelity image-to-mesh with material baked in. **≈85–120 s** per asset on |
| an RTX 5090, returning ≈15–17 MB `.glb` files that already carry shading. |
|
|
| > **What this repo is:** the Pixal3D pipeline at full precision — the three |
| > 1.3B transformers as ≈5.5 GB bf16 safetensors under `ckpts/`. Weights only, |
| > not a retrain. A **GGUF build of the same weights** (≈2.7 GB per |
| > transformer) lives in |
| > [`ChrisColeTech/Pixal3D-GGUF`](https://huggingface.co/ChrisColeTech/Pixal3D-GGUF). |
|
|
| --- |
|
|
| ## Samples |
|
|
| Single image in, textured mesh out. Source images were rendered with Z-Image |
| Turbo; the meshes are Blender Workbench renders of the raw `.glb` — no |
| cleanup, no retopology, no separate paint stage. |
|
|
| > **Note on these samples:** they were rendered from the **GGUF build** of |
| > these weights, not from the full-precision files in this repo. Timings and |
| > mesh sizes come from that run. The architecture and settings are identical; |
| > expect the full-precision path to differ only marginally, but the numbers |
| > below are not measured on it. |
|
|
| <table> |
| <tr><td><img src="https://huggingface.co/ChrisColeTech/Pixal3D/resolve/main/samples/src-robot.png" width="330" alt="source photograph of a retro robot toy"></td><td><img src="https://huggingface.co/ChrisColeTech/Pixal3D/resolve/main/samples/mesh-robot.png" width="330" alt="generated mesh with baked material"></td></tr> |
| <tr><td><sub>**source** — `a cute chunky retro robot toy standing upright, rounded metal body, simple friendly face, studio product photograph on a plain white background`</sub></td><td><sub>**mesh** — 17.1 MB `.glb`, generated in 119 s, seed 42</sub></td></tr> |
| <tr><td><img src="https://huggingface.co/ChrisColeTech/Pixal3D/resolve/main/samples/src-teapot.png" width="330" alt="source photograph of a porcelain teapot"></td><td><img src="https://huggingface.co/ChrisColeTech/Pixal3D/resolve/main/samples/mesh-teapot.png" width="330" alt="generated mesh with baked material"></td></tr> |
| <tr><td><sub>**source** — `an ornate ceramic teapot with a curved spout and handle, glazed blue and white porcelain, studio product photograph`</sub></td><td><sub>**mesh** — 15.5 MB `.glb`, generated in 85 s, seed 42</sub></td></tr> |
| </table> |
|
|
| ### Turntable |
|
|
| Four views, 90° apart. |
|
|
|  |
|
|
|  |
|
|
| The chest display and dial, the ear knobs and the boot flare all survive, and |
| the surface reads as brushed metal — material is produced with the geometry |
| rather than in a later pass. |
|
|
| ## Recommended settings |
|
|
| | Parameter | Production value | Meaning | |
| |---|---|---| |
| | `seed` | any | Deterministic per seed | |
| | input | one RGB(A) image | Centred subject, plain background | |
| | output | `.glb` | Mesh **with baked material** | |
|
|
| **Supported modes:** `img2mesh`, `txt2mesh`, and the `_textured` variants |
|
|
| ### Notes and gotchas |
|
|
| - **⚠ Meshes export rotated 180° relative to some other generators.** Pixal3D |
| puts the subject's front along the opposite axis from TripoSG/TRELLIS.2, so |
| a fixed camera that frames those correctly will render Pixal3D output from |
| behind. If you batch-render thumbnails, orbit the camera 180° (or check one |
| contact sheet per model before trusting a shared preset). |
| - **Material comes for free** — no separate paint stage needed, which is the |
| main reason to choose this over a geometry-only model. |
| - **Loads in a low-VRAM mode** by default, reporting ≈0 GiB resident after |
| load and streaming weights in as it runs. Generation still wants the card |
| largely to itself; sharing a GPU with another model-serving process |
| produces failures that look like crashes rather than clean OOM errors. |
| - **Sparse-conv backend** — logs `[SPARSE] Conv backend: flex_gemm; |
| Attention backend: flash_attn` on load and pulls a `valeoai/NAF` estimator |
| from `torch.hub` on first run, so a fresh machine needs network access. |
|
|
| --- |
|
|
| ## Files |
|
|
| | Path | Size | Role | |
| |---|---|---| |
| | `ckpts/slat_flow_img2shape_dit_1_3B_1024_bf16.safetensors` | 5.55 GB | shape DiT, 1024 latents | |
| | `ckpts/slat_flow_img2shape_dit_1_3B_512_bf16.safetensors` | 5.55 GB | shape DiT, 512 latents | |
| | `ckpts/slat_flow_imgshape2tex_dit_1_3B_1024_bf16.safetensors` | 5.55 GB | texture DiT | |
| | `ckpts/ss_flow_img_dit_1_3B_64_bf16.safetensors` | 5.36 GB | sparse-structure DiT | |
| | `ckpts/shape_dec_next_dc_f16c32_fp16.safetensors` | 948 MB | shape decoder | |
| | `ckpts/tex_dec_next_dc_f16c32_fp16.safetensors` | 948 MB | texture decoder | |
| | `ckpts/ss_dec_conv3d_16l8_fp16.safetensors` | 148 MB | sparse-structure decoder | |
|
|
| For roughly half the download and resident footprint, use the GGUF build: |
| [`ChrisColeTech/Pixal3D-GGUF`](https://huggingface.co/ChrisColeTech/Pixal3D-GGUF). |
|
|
| --- |
|
|
| ## Provenance |
|
|
| - **Upstream base model:** Pixal3D (TencentARC) |
| - **This build:** redistributed as-is at full precision. Weights are not retrained here. |
| - **License:** MIT, per upstream. |
|
|
