README.md

metadata

title: SAM 3.1 Concept Segmentation
emoji: 🎯
colorFrom: indigo
colorTo: blue
sdk: gradio
app_file: app.py
pinned: false
license: other
short_description: Language-driven segmentation with Meta SAM 3.1.

SAM 3.1 · Concept Segmentation

A live, language-driven segmentation demo built on Meta's Segment Anything Model 3.1. Type a short noun phrase (horse, saddle, person) and the model finds and segments every matching instance in the image — no boxes, no clicks, no retraining.

Segmentation is fully driven by language prompts — no retraining required.

What's inside

• Text-prompt segmentation via SAM 3.1's Promptable Concept Segmentation (PCS).
• Three result views in tabs — Overlay (semi-transparent, color-coded per instance), Mask only, and Original.
• Instant feedback — match count + inference latency shown after every run.
• Advanced mode — confidence threshold, optional boxes/scores, and a multiple-prompts runner that reuses a single vision pass for speed.
• Prompt history — a gallery of your recent prompts and their overlays.

Deploy on Hugging Face Spaces

1. Create a Space → SDK: Gradio. Upload app.py, requirements.txt, and this README.md (or push the repo).
2. Request model access. SAM 3 / 3.1 weights are gated. Open the model page (e.g. https://huggingface.co/facebook/sam3.1) and accept the license. Do the same for facebook/sam3 if you want the fallback.
3. Add your token. In Settings → Variables and secrets, add a secret named HF_TOKEN with a read token from https://huggingface.co/settings/tokens.
4. Pick hardware. SAM 3.1 is an ~848M-parameter GPU model.
   • ZeroGPU (free, recommended for public demos) — works out of the box; the app uses @spaces.GPU.
   • or a small GPU Space (e.g. T4 / A10G) for an always-on demo.
   • CPU works but is slow; fine only for smoke-testing.
5. Open the Space. The first request downloads the weights (cold start takes a bit); subsequent prompts are fast.

Run locally

pip install torch            # from https://pytorch.org for your platform/CUDA
pip install -r requirements.txt
export HF_TOKEN=hf_...        # token with access to the gated weights
python app.py

Configuration (environment variables)

Variable	Default	Purpose
MODEL_ID	facebook/sam3.1	Primary checkpoint to load.
FALLBACK_MODEL_ID	facebook/sam3	Used automatically if the primary fails to load.
HF_TOKEN	—	Access token for the gated SAM 3 / 3.1 weights.

To force SAM 3 instead of 3.1, set MODEL_ID=facebook/sam3.

Notes & tips

• Short noun phrases win. PCS is tuned for concepts like horse or saddle. Long descriptive phrases are less reliable — for reins, prefer reins or bridle over "object used for riding control." The example dropdown keeps the descriptive phrase so you can see the difference for yourself.
• Threshold. Lower it (Advanced) to surface more instances; raise it to keep only high-confidence matches.
• Speed. On GPU the forward pass runs in bfloat16. The multi-prompt runner computes vision features once and reuses them across prompts in a single call.
• Example images. Drop files into an examples/ folder (horse.jpg, street.jpg, kitchen.jpg) to enable one-click examples. The app runs fine without them.

About the model

Built on Meta's Segment Anything Model 3.1 (released March 2026), which extends SAM 3's Promptable Concept Segmentation with the Object Multiplex tracker for faster multi-object video. This demo uses the image PCS path through 🤗 Transformers (Sam3Model / Sam3Processor). The SAM 3 / 3.1 weights are distributed under Meta's SAM License — review it before any production use.