---
license: mit
tags:
  - 3d
  - scene-completion
  - scene-generation
  - flow-matching
  - tsdf
  - sparse-transformer
  - controlnet
---

# Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow

[![Project Page](https://img.shields.io/badge/Project-Page-blue)](https://quan-meng.github.io/projects/seen2scene/)
[![arXiv](https://img.shields.io/badge/arXiv-Paper-red)](https://arxiv.org/abs/2603.28548)
[![Code](https://img.shields.io/badge/GitHub-Code-black?logo=github)](https://github.com/quan-meng/seen2scene)
[![YouTube](https://img.shields.io/badge/YouTube-Video-FF0000?logo=youtube)](https://www.youtube.com/watch?v=5qJYLjMsJe8)

Pretrained checkpoints for **Seen2Scene**, the first flow matching-based approach that trains directly on
incomplete, real-world 3D scans for scene completion and generation. Seen2Scene introduces
**visibility-guided flow matching**, which explicitly masks out unknown regions in real scans, enabling
effective learning from real-world, partial observations.

Code, installation instructions, and inference commands live in the
[GitHub repository](https://github.com/quan-meng/seen2scene).

## Checkpoints

| Component | Role | Size | Path in this repo |
|---|---|---|---|
| VAE (`AE_LOG`) | Encodes TSDF scenes into sparse latents | 925 MB | `experiments/auto_encoder/2025-12-19_01-23-28-525/checkpoint/vxl_0_011_last.ckpt` |
| Flow Matching generator (`GEN_LOG`) | Layout-conditioned scene generation | 4.5 GB | `experiments/auto_encoder/2025-12-19_01-23-28-525/generator/2026-02-23_16-22-25-152/checkpoint/vxl_0_011_last.ckpt` |
| ControlNet (`CONTROL_LOG`) | Partial-scan completion | 1.2 GB | `experiments/auto_encoder/2025-12-19_01-23-28-525/generator/2026-02-23_16-22-25-152/control/2026-02-26_14-01-23-930/checkpoint/vxl_0_011_last.ckpt` |

The three run folders correspond to the `AE_LOG`, `GEN_LOG`, and `CONTROL_LOG` arguments of the
inference commands:

```text
AE_LOG      = 2025-12-19_01-23-28-525
GEN_LOG     = 2026-02-23_16-22-25-152
CONTROL_LOG = 2026-02-26_14-01-23-930
```

- **Generation** requires the VAE and Flow Matching generator checkpoints.
- **Completion** requires the VAE, Flow Matching generator, and ControlNet checkpoints.

## Download

The repository mirrors the training log hierarchy, so downloading into the repository root places
every checkpoint where the code expects it — no renaming or moving required.

```bash
pip install -U "huggingface_hub[cli]"

# From the root of your Seen2Scene checkout:
hf download MQ66/seen2scene --local-dir .
```

To fetch a single checkpoint, for example the VAE only:

```bash
hf download MQ66/seen2scene \
  experiments/auto_encoder/2025-12-19_01-23-28-525/checkpoint/vxl_0_011_last.ckpt \
  --local-dir .
```

Or from Python:

```python
from huggingface_hub import snapshot_download

snapshot_download(repo_id="MQ66/seen2scene", local_dir=".")
```

## Usage

After downloading, run inference from the repository root (see the
[GitHub README](https://github.com/quan-meng/seen2scene) for the full option list):

```bash
# Partial-scan completion with the ControlNet.
python -m seen2scene.main control task:completion \
  --ae-log 2025-12-19_01-23-28-525 \
  --gen-log 2026-02-23_16-22-25-152 \
  --ckpt-path 2026-02-26_14-01-23-930 \
  --src-key tsdf_p_0.1 \
  --latent-key tsdf_p_1.0 \
  --task.num-samples 10 \
  --task.export-as bbox mesh volume \
  --slurm.cluster local

# Layout-conditioned patch generation with the Flow Matching generator.
python -m seen2scene.main generator task:generation \
  --ae-log 2025-12-19_01-23-28-525 \
  --ckpt-path 2026-02-23_16-22-25-152 \
  --latent-key tsdf_p_1.0 \
  --task.num-samples 10 \
  --task.export-as bbox mesh \
  --slurm.cluster local
```

## Training data

The models were trained on [3D-FRONT](https://tianchi.aliyun.com/specials/promotion/alibaba-3d-scene-dataset),
[ScanNet++](https://kaldir.vc.in.tum.de/scannetpp/), and [ARKitScenes](https://github.com/apple/ARKitScenes).
These datasets carry their own licenses and terms of use, which apply independently of the license of
these weights.

## Citation

```bibtex
@misc{meng2026seen2scene,
      title={Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow},
      author={Quan Meng and Yujin Chen and Lei Li and Matthias Nießner and Angela Dai},
      year={2026},
      eprint={2603.28548},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2603.28548},
}
```

## License

Released under the [MIT License](https://github.com/quan-meng/seen2scene/blob/main/LICENSE).
Third-party components retain their respective licenses; see the
[Third-Party Notices](https://github.com/quan-meng/seen2scene/blob/main/THIRD_PARTY_NOTICES.md).