README.md

---
title: SAM 3.1 Concept Segmentation
emoji: 🎯
colorFrom: indigo
colorTo: blue
sdk: gradio
app_file: app.py
pinned: false
license: other
short_description: Language-driven segmentation with Meta SAM 3.1.
---

# SAM 3.1 · Concept Segmentation

A live, **language-driven** segmentation demo built on Meta's **Segment Anything
Model 3.1**. Type a short noun phrase (`horse`, `saddle`, `person`) and the model
finds and segments **every matching instance** in the image — no boxes, no clicks,
no retraining.

> Segmentation is fully driven by language prompts — no retraining required.

## What's inside

- **Text-prompt segmentation** via SAM 3.1's Promptable Concept Segmentation (PCS).
- **Three result views** in tabs — *Overlay* (semi-transparent, color-coded per
  instance), *Mask only*, and *Original*.
- **Instant feedback** — match count + inference latency shown after every run.
- **Advanced mode** — confidence threshold, optional boxes/scores, and a
  **multiple-prompts** runner that reuses a single vision pass for speed.
- **Prompt history** — a gallery of your recent prompts and their overlays.

## Deploy on Hugging Face Spaces

1. **Create a Space** → SDK: **Gradio**. Upload `app.py`, `requirements.txt`,
   and this `README.md` (or push the repo).
2. **Request model access.** SAM 3 / 3.1 weights are gated. Open the model page
   (e.g. <https://huggingface.co/facebook/sam3.1>) and accept the license. Do the
   same for `facebook/sam3` if you want the fallback.
3. **Add your token.** In **Settings → Variables and secrets**, add a secret named
   **`HF_TOKEN`** with a read token from <https://huggingface.co/settings/tokens>.
4. **Pick hardware.** SAM 3.1 is an ~848M-parameter GPU model.
   - **ZeroGPU** (free, recommended for public demos) — works out of the box; the
     app uses `@spaces.GPU`.
   - or a small **GPU Space** (e.g. T4 / A10G) for an always-on demo.
   - CPU works but is slow; fine only for smoke-testing.
5. **Open the Space.** The first request downloads the weights (cold start takes a
   bit); subsequent prompts are fast.

### Run locally

```bash
pip install torch            # from https://pytorch.org for your platform/CUDA
pip install -r requirements.txt
export HF_TOKEN=hf_...        # token with access to the gated weights
python app.py
```

## Configuration (environment variables)

| Variable             | Default          | Purpose                                              |
|----------------------|------------------|------------------------------------------------------|
| `MODEL_ID`           | `facebook/sam3.1`| Primary checkpoint to load.                          |
| `FALLBACK_MODEL_ID`  | `facebook/sam3`  | Used automatically if the primary fails to load.     |
| `HF_TOKEN`           | —                | Access token for the gated SAM 3 / 3.1 weights.      |

To force SAM 3 instead of 3.1, set `MODEL_ID=facebook/sam3`.

## Notes & tips

- **Short noun phrases win.** PCS is tuned for concepts like `horse` or `saddle`.
  Long descriptive phrases are less reliable — for reins, prefer `reins` or
  `bridle` over *"object used for riding control."* The example dropdown keeps the
  descriptive phrase so you can see the difference for yourself.
- **Threshold.** Lower it (Advanced) to surface more instances; raise it to keep
  only high-confidence matches.
- **Speed.** On GPU the forward pass runs in bfloat16. The multi-prompt runner
  computes vision features once and reuses them across prompts in a single call.
- **Example images.** Drop files into an `examples/` folder (`horse.jpg`,
  `street.jpg`, `kitchen.jpg`) to enable one-click examples. The app runs fine
  without them.

## About the model

Built on Meta's Segment Anything Model 3.1 (released March 2026), which extends
SAM 3's Promptable Concept Segmentation with the *Object Multiplex* tracker for
faster multi-object video. This demo uses the **image** PCS path through 🤗
Transformers (`Sam3Model` / `Sam3Processor`). The SAM 3 / 3.1 weights are
distributed under Meta's SAM License — review it before any production use.