README.md

---
license: other
pipeline_tag: mask-generation
library_name: pytorch
base_model: facebook/sam3
tags:
- sam3
- lora
- mask-generation
- image-segmentation
- semantic-segmentation
- breast-ultrasound
- medical-imaging
---

# SAM3 LoRA Breast Lesion

Portable SAM3 semantic-text segmentation bundle fine-tuned on the BUSCoT
`trainval` split. The training prompt was `breast lesion`.

## Important Architecture Note

Inference requires two checkpoints:

1. `model/sam3_base.pt`: original SAM3 base checkpoint used to construct SAM3.
2. `model/best_model.pt`: fine-tuned LoRA/native-head checkpoint.

`best_model.pt` is not a self-contained model. The bundled wrapper first builds
the original SAM3 architecture from `sam3_base.pt`, injects LoRA modules, and
then loads `best_model.pt`.

The bundle also includes:

- `runtime/sam3_repo/sam3/`: required SAM3 Python source.
- `scripts/sam3_decoder_experiment_lib.py`: custom LoRA/native-decoder model.
- `scripts/sam3_buscot_runner.py`: single-image prediction wrapper.

No paths from the original training server are required for inference.

## Model Configuration

| Item | Value |
|---|---|
| Prompt type | Semantic text |
| Training prompt | `breast lesion` |
| Input size | 512 |
| LoRA rank | 8 |
| LoRA alpha | 16 |
| Default threshold | 0.5 |
| Validation-selected threshold | 0.6 |

BUSCoT test performance at threshold 0.5:

| Mean Dice | Mean IoU | Mean HD95 | Mean ASSD |
|---:|---:|---:|---:|
| 0.8782 | 0.8130 | 27.35 | 10.57 |

## Package Layout

```text
model/
  sam3_base.pt                 original SAM3 base checkpoint
  best_model.pt                fine-tuned checkpoint
  config.json
  thresholds.json
runtime/sam3_repo/
  sam3/                        SAM3 source and tokenizer asset
scripts/
  infer_single_image.py        portable single-image inference
  sam3_buscot_runner.py
  sam3_decoder_experiment_lib.py
  evaluate_sam3_test_text_prompt_segmentation.py
  evaluate_sam3_lora_multiprompt_sensitivity.py
  analyze_sam3_prompt_robust_vs_nonrobust.py
  train_sam3_decoder_segmentation.py
  run_single_prompt_eval.sh
  run_5prompt_qc.sh
metrics/
examples/
requirements.txt
```

## Environment

Python 3.10 is recommended. Install dependencies into an environment that has a
CUDA-compatible PyTorch build:

```bash
cd "/path/to/SAM3 LoRA Breast Lesion"
python -m pip install -r requirements.txt
```

If a compatible SAM3 environment already exists, activate it and skip the
installation command.

## Single-Image Inference

Run from any directory:

```bash
CUDA_VISIBLE_DEVICES=0 python \
  "/path/to/SAM3 LoRA Breast Lesion/scripts/infer_single_image.py" \
  --image /path/to/breast_ultrasound.png \
  --output /path/to/predicted_mask.png \
  --prompt "breast lesion"
```

This inference path does not use a GT mask or bbox.

To use an external SAM3 base checkpoint instead of the bundled copy:

```bash
SAM3_CHECKPOINT=/path/to/sam3.pt CUDA_VISIBLE_DEVICES=0 python \
  "/path/to/SAM3 LoRA Breast Lesion/scripts/infer_single_image.py" \
  --image /path/to/image.png \
  --output /path/to/mask.png
```

## Batch Evaluation With GT

The input CSV must contain at least:

```text
sample_id,original_image_path,gt_mask_path,gt_mask_bbox,has_original_image,has_gt_mask
```

Run:

```bash
cd "/path/to/SAM3 LoRA Breast Lesion"
PYTHON=/path/to/python bash scripts/run_single_prompt_eval.sh \
  /path/to/index.csv /path/to/output 0
```

Arguments are `INDEX_CSV`, `OUTPUT_DIR`, and `GPU`.

## Five-Prompt QC

```bash
cd "/path/to/SAM3 LoRA Breast Lesion"
PYTHON=/path/to/python bash scripts/run_5prompt_qc.sh \
  /path/to/index.csv /path/to/output 0
```

The prompts are:

```text
breast lesion
breast tumor
breast mass
breast nodule
ultrasound breast lesion
```

The QC pipeline calculates pairwise predicted-mask Dice and uses minimum
pairwise Dice as a deployable consistency signal. GT masks are needed only for
offline accuracy evaluation.

## Optional Retraining

Retraining requires a compatible dataset CSV with `train`, `val`, and `test`
rows. It is not needed for inference:

```bash
cd "/path/to/SAM3 LoRA Breast Lesion"
PYTHON=/path/to/python BATCH_SIZE=8 bash scripts/train_sam3_buscot_lora.sh \
  /path/to/dataset.csv /path/to/new_output 0
```

## Troubleshooting

Verify all bundled files:

```bash
cd "/path/to/SAM3 LoRA Breast Lesion"
sha256sum -c SHA256SUMS
```

Verify imports without loading the checkpoints:

```bash
cd /tmp
PYTHONPATH="/path/to/SAM3 LoRA Breast Lesion/scripts:/path/to/SAM3 LoRA Breast Lesion/runtime/sam3_repo" \
python -c "from sam3_buscot_runner import SAM3BuscotPredictor; print('imports OK')"
```

Common issues:

- `No module named sam3_decoder_experiment_lib`: bundle is incomplete or an old
  copy is being used.
- `No module named sam3`: `runtime/sam3_repo/sam3/` is missing.
- Missing `sam3_base.pt`: the original base checkpoint was not transferred.
- CUDA/PyTorch errors: install a PyTorch build compatible with the target
  server's CUDA driver.
- Package version errors: use the provided `requirements.txt` and Python 3.10.

## License

The bundled SAM3 source and base checkpoint remain subject to the SAM3 license
included at `runtime/sam3_repo/LICENSE`.