license: apache-2.0
pipeline_tag: image-segmentation
tags:
- sam3
- vision-language
- segmentation
- efficient-sam
SAM3-LiteText
SAM3-LiteText is a lightweight text encoding framework for vision-language segmentation, introduced in the paper SAM3-LiteText: An Anatomical Study of the SAM3 Text Encoder for Efficient Vision-Language Segmentation.
Introduction
Vision-language segmentation models like SAM3 enable flexible, prompt-driven visual grounding but often rely on large text encoders designed for open-ended language understanding. SAM3-LiteText addresses this by replacing the original SAM3 text encoder with a compact MobileCLIP student optimized via knowledge distillation. This approach reduces text encoder parameters by up to 88% and significantly lowers the memory footprint while maintaining segmentation performance comparable to the original model.
Resources
- Paper: SAM3-LiteText: An Anatomical Study of the SAM3 Text Encoder for Efficient Vision-Language Segmentation
- Code: GitHub Repository (sam3_litetext branch)
- Project Page: EfficientSAM3
Sample Usage
The following example demonstrates how to perform inference with a text prompt using an EfficientSAM3 model variant with a distilled MobileCLIP text encoder:
from sam3.model_builder import build_efficientsam3_image_model
from sam3.model.sam3_image_processor import Sam3Processor
# Load model with distilled text encoder
model = build_efficientsam3_image_model(
checkpoint_path="efficient_sam3_tinyvit_m_mobileclip_s1.pt",
backbone_type="tinyvit",
model_name="11m",
text_encoder_type="MobileCLIP-S1"
)
# Process image and predict with text prompt
processor = Sam3Processor(model)
inference_state = processor.set_image(image)
inference_state = processor.set_text_prompt(prompt="shoe", state=inference_state)
masks = inference_state["masks"]
scores = inference_state["scores"]
print(f"Found {len(scores)} masks. Scores: {scores}")
Citation
If you use SAM3-LiteText or the EfficientSAM3 framework in your research, please cite:
@misc{zeng2025efficientsam3progressivehierarchicaldistillation,
title={EfficientSAM3: Progressive Hierarchical Distillation for Video Concept Segmentation from SAM1, 2, and 3},
author={Chengxi Zeng and Yuxuan Jiang and Gao Ge and Shuai Wang and Fan Aaron Zhang},
year={2025},
eprint={2511.15833},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2511.15833},
}