| --- |
| license: apache-2.0 |
| pipeline_tag: image-segmentation |
| tags: |
| - sam3 |
| - vision-language |
| - segmentation |
| - efficient-sam |
| --- |
| |
| # SAM3-LiteText |
|
|
| SAM3-LiteText is a lightweight text encoding framework for vision-language segmentation, introduced in the paper [SAM3-LiteText: An Anatomical Study of the SAM3 Text Encoder for Efficient Vision-Language Segmentation](https://huggingface.co/papers/2602.12173). |
|
|
| ## Introduction |
|
|
| Vision-language segmentation models like SAM3 enable flexible, prompt-driven visual grounding but often rely on large text encoders designed for open-ended language understanding. SAM3-LiteText addresses this by replacing the original SAM3 text encoder with a compact **MobileCLIP** student optimized via knowledge distillation. This approach reduces text encoder parameters by up to 88% and significantly lowers the memory footprint while maintaining segmentation performance comparable to the original model. |
|
|
| ## Resources |
|
|
| - **Paper:** [SAM3-LiteText: An Anatomical Study of the SAM3 Text Encoder for Efficient Vision-Language Segmentation](https://huggingface.co/papers/2602.12173) |
| - **Code:** [GitHub Repository (sam3_litetext branch)](https://github.com/SimonZeng7108/efficientsam3/tree/sam3_litetext) |
| - **Project Page:** [EfficientSAM3](https://simonzeng7108.github.io/efficientsam3/) |
|
|
| ## Sample Usage |
|
|
| The following example demonstrates how to perform inference with a text prompt using an EfficientSAM3 model variant with a distilled MobileCLIP text encoder: |
|
|
| ```python |
| from sam3.model_builder import build_efficientsam3_image_model |
| from sam3.model.sam3_image_processor import Sam3Processor |
| |
| # Load model with distilled text encoder |
| model = build_efficientsam3_image_model( |
| checkpoint_path="efficient_sam3_tinyvit_m_mobileclip_s1.pt", |
| backbone_type="tinyvit", |
| model_name="11m", |
| text_encoder_type="MobileCLIP-S1" |
| ) |
| |
| # Process image and predict with text prompt |
| processor = Sam3Processor(model) |
| inference_state = processor.set_image(image) |
| inference_state = processor.set_text_prompt(prompt="shoe", state=inference_state) |
| |
| masks = inference_state["masks"] |
| scores = inference_state["scores"] |
| print(f"Found {len(scores)} masks. Scores: {scores}") |
| ``` |
|
|
| ## Citation |
|
|
| If you use SAM3-LiteText or the EfficientSAM3 framework in your research, please cite: |
|
|
| ```bibtex |
| @misc{zeng2025efficientsam3progressivehierarchicaldistillation, |
| title={EfficientSAM3: Progressive Hierarchical Distillation for Video Concept Segmentation from SAM1, 2, and 3}, |
| author={Chengxi Zeng and Yuxuan Jiang and Gao Ge and Shuai Wang and Fan Aaron Zhang}, |
| year={2025}, |
| eprint={2511.15833}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CV}, |
| url={https://arxiv.org/abs/2511.15833}, |
| } |
| ``` |
