metadata
license: mit
pipeline_tag: other
tags:
- 3d
- segmentation
SegviGen: Repurposing 3D Generative Model for Part Segmentation
SegviGen is a framework for 3D part segmentation that leverages the rich 3D structural and textural knowledge encoded in large-scale 3D generative models. It learns to predict part-indicative colors while reconstructing geometry, and unifies three settings in one architecture: interactive part segmentation, full segmentation, and 2D segmentation map–guided full segmentation with arbitrary granularity.
- Paper: SegviGen: Repurposing 3D Generative Model for Part Segmentation
- Project Page: https://fenghora.github.io/SegviGen-Page/
- Repository: https://github.com/Nelipot-Lee/SegviGen
- Demo: Hugging Face Space
Features
- Repurposed 3D Generative Priors for Data Efficiency: Reuses knowledge from native 3D generative models, requiring only 0.32% training data.
- Unified and Flexible Settings: Supports interactive, full, and 2D-guided segmentation in a single architecture.
- State-of-the-Art Accuracy: Delivers significant improvements in IoU over existing methods like P3-SAM.
Usage
For installation, please refer to the official GitHub repository.
Interactive part-segmentation
python inference_interactive.py \
--ckpt_path path/to/interactive_seg.ckpt \
--glb ./data_toolkit/assets/example.glb \
--input_vxz ./data_toolkit/assets/input.vxz \
--transforms ./data_toolkit/transforms.json \
--img ./data_toolkit/assets/img.png \
--export_glb ./data_toolkit/assets/output.glb \
--input_vxz_points 388 448 392
Full segmentation
python inference_full.py \
--ckpt_path path/to/full_seg.ckpt \
--glb ./data_toolkit/assets/example.glb \
--input_vxz ./data_toolkit/assets/input.vxz \
--transforms ./data_toolkit/transforms.json \
--img ./data_toolkit/assets/img.png \
--export_glb ./data_toolkit/assets/output.glb
Full segmentation with 2D guidance
python inference_full.py \
--ckpt_path path/to/full_seg_w_2d_map.ckpt \
--glb ./data_toolkit/assets/example.glb \
--input_vxz ./data_toolkit/assets/input.vxz \
--img ./data_toolkit/assets/full_seg_w_2d_map/2d_map.png \
--export_glb ./data_toolkit/assets/output.glb \
--two_d_map
Citation
@article{li2026segvigen,
title = {SegviGen: Repurposing 3D Generative Model for Part Segmentation},
author = {Lin Li and Haoran Feng and Zehuan Huang and Haohua Chen and Wenbo Nie and Shaohua Hou and Keqing Fan and Pan Hu and Sheng Wang and Buyu Li and Lu Sheng},
journal = {arXiv preprint arXiv:2603.16869},
year = {2026}
}