File size: 3,145 Bytes
5bd7ff9 a2c0c0d 5bd7ff9 a2c0c0d 5bd7ff9 0591151 5bd7ff9 a2c0c0d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | ---
base_model: microsoft/TRELLIS-image-large
language: en
license: mit
pipeline_tag: image-to-3d
tags:
- multi-view
- 3d-reconstruction
- camera-pose-estimation
- safetensors
---
# GenSfM pretrained models
[Paper](https://arxiv.org/abs/2610.35522) · [Code](https://github.com/dotb1ue/GenSfm) · [Project page](https://dotb1ue.github.io/GenSfm/)
GenSfM jointly reconstructs an object and its cameras from two to four foreground-masked views. Four views are recommended. This repository contains the complete reconstruction weights for the native two-stage pipeline.
## Usage
Install the GenSfM code and its inference dependencies first. From the code repository, run the multi-view entrypoint:
```bash
python example_multi_image.py \
--pretrained dotblue/GenSfm \
--input-dir /path/to/object-views \
--output-dir outputs/my-object
```
The same argument also accepts a local model directory. Use `--revision` to select a specific Hub tag or commit.
```python
from trellis.pipelines import GenSfMPipeline
pipeline = GenSfMPipeline.from_pretrained("dotblue/GenSfm").cuda()
result = pipeline.run(["00.png", "01.png", "02.png", "03.png"], seed=1)
mesh = result["mesh"][0]
gaussians = result["gaussian"][0]
extrinsics, intrinsics = result["extrinsics"], result["intrinsics"]
```
Inputs are ordered paths to RGBA images of the same rigid object. For photographs without alpha, add `--remove-background` to the CLI or pass `remove_background=True` to `run`. Camera intrinsics are normalized complete matrices, and extrinsics use OpenCV world-to-camera coordinates in the model's Z-up frame. The CLI exports a textured GLB, Gaussians, camera matrices, inputs and reprojections.
DINOv2 (`dinov2_vitl14_reg`) is obtained separately through Torch Hub. Optional background removal uses rembg's own model. Both need a first-use download or an existing cache.
## Contents
`pipeline.json` maps six runtime roles to model prefixes and records both samplers, latent normalization and input preprocessing. Each prefix has a `.json` architecture file and a `.safetensors` state dictionary.
| Prefix under `ckpts/` | Purpose |
| --- | --- |
| `stage1_joint` | Native joint shape and per-view camera flow model |
| `stage2_slat` | Trained ConvNeXt geometry and appearance flow model |
| `uv_decoder` | Trained UV representation decoder |
| `ss_decoder` | TRELLIS occupancy decoder |
| `slat_decoder_gs` | TRELLIS Gaussian decoder |
| `slat_decoder_mesh` | TRELLIS mesh decoder |
| `slat_encoder` | TRELLIS conditioning encoder loaded internally by stage 2 |
No training directories or machine-specific paths are needed. The bundle includes inference weights; full training resume state is not included.
## License and acknowledgements
The model weights are released under the [MIT License](LICENSE).
The sparse-structure decoder, SLat Gaussian/mesh decoders and SLat encoder originate from [microsoft/TRELLIS-image-large](https://huggingface.co/microsoft/TRELLIS-image-large). Their original MIT notice is preserved in `LICENSE_TRELLIS`. DINOv2 and optional background-removal weights are external dependencies and are not redistributed here. |