| --- |
| base_model: microsoft/TRELLIS-image-large |
| language: en |
| license: mit |
| pipeline_tag: image-to-3d |
| tags: |
| - multi-view |
| - 3d-reconstruction |
| - camera-pose-estimation |
| - safetensors |
| --- |
| |
| # GenSfM pretrained models |
|
|
| [Paper](https://arxiv.org/abs/2610.35522) · [Code](https://github.com/dotb1ue/GenSfm) · [Project page](https://dotb1ue.github.io/GenSfm/) |
|
|
| GenSfM jointly reconstructs an object and its cameras from two to four foreground-masked views. Four views are recommended. This repository contains the complete reconstruction weights for the native two-stage pipeline. |
|
|
| ## Usage |
|
|
| Install the GenSfM code and its inference dependencies first. From the code repository, run the multi-view entrypoint: |
|
|
| ```bash |
| python example_multi_image.py \ |
| --pretrained dotblue/GenSfm \ |
| --input-dir /path/to/object-views \ |
| --output-dir outputs/my-object |
| ``` |
|
|
| The same argument also accepts a local model directory. Use `--revision` to select a specific Hub tag or commit. |
|
|
| ```python |
| from trellis.pipelines import GenSfMPipeline |
| |
| pipeline = GenSfMPipeline.from_pretrained("dotblue/GenSfm").cuda() |
| result = pipeline.run(["00.png", "01.png", "02.png", "03.png"], seed=1) |
| mesh = result["mesh"][0] |
| gaussians = result["gaussian"][0] |
| extrinsics, intrinsics = result["extrinsics"], result["intrinsics"] |
| ``` |
|
|
| Inputs are ordered paths to RGBA images of the same rigid object. For photographs without alpha, add `--remove-background` to the CLI or pass `remove_background=True` to `run`. Camera intrinsics are normalized complete matrices, and extrinsics use OpenCV world-to-camera coordinates in the model's Z-up frame. The CLI exports a textured GLB, Gaussians, camera matrices, inputs and reprojections. |
|
|
| DINOv2 (`dinov2_vitl14_reg`) is obtained separately through Torch Hub. Optional background removal uses rembg's own model. Both need a first-use download or an existing cache. |
|
|
| ## Contents |
|
|
| `pipeline.json` maps six runtime roles to model prefixes and records both samplers, latent normalization and input preprocessing. Each prefix has a `.json` architecture file and a `.safetensors` state dictionary. |
|
|
| | Prefix under `ckpts/` | Purpose | |
| | --- | --- | |
| | `stage1_joint` | Native joint shape and per-view camera flow model | |
| | `stage2_slat` | Trained ConvNeXt geometry and appearance flow model | |
| | `uv_decoder` | Trained UV representation decoder | |
| | `ss_decoder` | TRELLIS occupancy decoder | |
| | `slat_decoder_gs` | TRELLIS Gaussian decoder | |
| | `slat_decoder_mesh` | TRELLIS mesh decoder | |
| | `slat_encoder` | TRELLIS conditioning encoder loaded internally by stage 2 | |
|
|
| No training directories or machine-specific paths are needed. The bundle includes inference weights; full training resume state is not included. |
|
|
| ## License and acknowledgements |
|
|
| The model weights are released under the [MIT License](LICENSE). |
|
|
| The sparse-structure decoder, SLat Gaussian/mesh decoders and SLat encoder originate from [microsoft/TRELLIS-image-large](https://huggingface.co/microsoft/TRELLIS-image-large). Their original MIT notice is preserved in `LICENSE_TRELLIS`. DINOv2 and optional background-removal weights are external dependencies and are not redistributed here. |
