LegoOcc: Monocular Open Vocabulary Occupancy Prediction for Indoor Scenes

CVPR 2026 Oral · Paper · Project Page

LegoOcc predicts open-vocabulary 3D semantic occupancy from a single RGB image. It represents the scene as language-embedded 3D Gaussians and voxelizes them with an opacity-aware Poisson Gaussian-to-occupancy operator, producing a 60×60×36 grid of 8 cm voxels. Each voxel is labeled by comparing its CLIP-aligned feature with text prompts, so you can query any vocabulary. The model was trained on OccScanNet with geometry-only supervision.

Usage

Install the self-contained inference package from the demo/ folder of the GitHub repo (pip install -r requirements.txt). It needs PyTorch with CUDA and nvcc. The CUDA op is compiled automatically the first time you load the model. Custom vocabularies also need pip install open_clip_torch.

from legoocc_hub import LegoOcc

model = LegoOcc.from_pretrained("JuIvyy/LegoOcc").cuda().eval()
out = model.predict("room.jpg", intrinsics=K)                                   # 11 OccScanNet classes
out = model.predict("room.jpg", vocabulary=["sofa", "lamp", "floor", "wall"])  # any vocabulary

out["occupancy"]    # (60, 60, 36): 0 = empty, k = out["class_names"][k]

Command-line demo (saves the occupancy as a colored .ply point cloud):

python demo.py --model JuIvyy/LegoOcc --image examples/scene0472_00_00061.jpg --vocab "sink,stove,cabinet,floor,wall"

The default vocabulary (ceiling, floor, wall, window, chair, bed, sofa, table, television, furniture, objects) is stored with the weights as precomputed CLIP ViT-B/16 text features.

License

For non-commercial research use only. The weights are derived from Depth-Anything-V2-Base (CC-BY-NC-4.0), and the Gaussian aggregation CUDA op builds on Inria's GRAPHDECO code.

Citation

@inproceedings{zhou2026monocular,
  title={Monocular open vocabulary occupancy prediction for indoor scenes},
  author={Zhou, Changqing and Luo, Yueru and Zhang, Han and Jiang, Zeyu and Chen, Changhao},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={21627--21637},
  year={2026}
}