File size: 3,512 Bytes
877f76c 70540b5 877f76c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | ---
license: apache-2.0
language:
- en
pipeline_tag: image-segmentation
tags:
- pytorch
- sam2
- microcrack-segmentation
- video-segmentation
---
# MP-SAM2
Fine-tuned models for **A Multi-Prompt Transfer Learning Framework Based on
Segment Anything Model 2 for Microcrack Segmentation in Engineered Cementitious
Composite Images and Videos**.
[Source and installation](https://github.com/cclyr/MP-SAM2) ·
[Training settings](https://github.com/cclyr/MP-SAM2/tree/main/configs) ·
[Model architecture](https://github.com/cclyr/MP-SAM2/blob/main/sam2/configs/sam2_hiera_l.yaml)
MP-SAM2 combines an automatic threshold mask, SP-Box, and EDT positive/negative
point prompts with progressive SAM2 transfer learning. The GitHub repository
provides training, image inference, video propagation, and evaluation scripts.
## Checkpoints
| File | Training images |
|---|---:|
| [mp_sam2_5.torch](https://huggingface.co/ccllr/MP-SAM2/resolve/main/mp_sam2_5.torch) | 5 |
| [mp_sam2_10.torch](https://huggingface.co/ccllr/MP-SAM2/resolve/main/mp_sam2_10.torch) | 10 |
| [mp_sam2_50.torch](https://huggingface.co/ccllr/MP-SAM2/resolve/main/mp_sam2_50.torch) | 50 |
| [mp_sam2_full.torch](https://huggingface.co/ccllr/MP-SAM2/resolve/main/mp_sam2_full.torch) | 186 (full data) |
Each file is approximately 898 MB and contains the complete model state for the
bundled `sam2_hiera_l.yaml` architecture. Each file is an individual trained
model; the paper's few-shot results summarize repeated training runs.
A 20-image checkpoint is not included. `SHA256SUMS` lists the file hashes.
## Usage
Follow the [source installation instructions](https://github.com/cclyr/MP-SAM2#installation).
From the source repository root:
```bash
python scripts/download_checkpoints.py --variant full
python scripts/infer_image.py \
--input path/to/image.png \
--checkpoint checkpoints/mp_sam2_full.torch \
--config configs/prompt.yaml \
--output outputs/image
```
Use `--variant 5`, `10`, `50`, or `all` for other checkpoints. Direct downloads
are also linked above. The download helper verifies MP-SAM2 file checksums.
Training and video inference also need Meta's original
[SAM2.1 Hiera Large checkpoint](https://dl.fbaipublicfiles.com/segment_anything_2/092824/sam2.1_hiera_large.pt).
Download it into `checkpoints/` with:
```bash
python scripts/download_checkpoints.py --variant full --include-base
```
The transfer loader selects matching SAM2.1 tensors for the bundled MP-SAM2
architecture, excluding SAM2.1-specific spatial and pointer-projection heads.
Use this repository's runtime and configuration with these weights. The files
are custom PyTorch state dictionaries loaded by the MP-SAM2 scripts; a
Transformers `AutoModel` interface is not supplied.
`configs/prompt.yaml` controls automatic prompts for image inference and the
first video frame. `configs/fewshot.yaml` and `configs/full.yaml` control the
respective training schedules. Full instructions, data layout, and evaluation
commands are in the [README](https://github.com/cclyr/MP-SAM2#readme).
## Scope and license
The intended application is microcrack segmentation in engineered cementitious
composite images and laboratory videos.
The original Meta checkpoint is obtained from upstream.
The four fine-tuned files are distributed under Apache License 2.0. See `LICENSE`
and the upstream SAM2 attribution in `NOTICE`. Citation metadata is provided in
the source repository's [CITATION.cff](https://github.com/cclyr/MP-SAM2/blob/main/CITATION.cff).
|