Instructions to use infosave/SAM3-cmf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- cortiq
How to use infosave/SAM3-cmf with cortiq:
# one Rust binary, no additional dependencies cargo install cortiq-cli # or a prebuilt binary from github.com/infosave2007/cmf/releases hf download infosave/SAM3-cmf --include "*.cmf" --local-dir . ls *.cmf # some repos ship more than one quantization
cortiq run FILE.cmf --prompt "What is the capital of France?"
cortiq serve FILE.cmf --port 8080 # OpenAI-compatible server
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: sam-license
|
| 4 |
+
library_name: cortiq
|
| 5 |
+
base_model:
|
| 6 |
+
- facebook/sam3
|
| 7 |
+
base_model_relation: quantized
|
| 8 |
+
pipeline_tag: image-segmentation
|
| 9 |
+
tags:
|
| 10 |
+
- cmf
|
| 11 |
+
- cortiq
|
| 12 |
+
- image-segmentation
|
| 13 |
+
- sam3
|
| 14 |
+
- video-tracking
|
| 15 |
+
- q8_2f
|
| 16 |
+
- rust
|
| 17 |
+
language:
|
| 18 |
+
- en
|
| 19 |
+
- ru
|
| 20 |
+
- zh
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
# SAM3 CMF
|
| 24 |
+
|
| 25 |
+
Native, single-file CMF packaging of
|
| 26 |
+
[facebook/sam3](https://huggingface.co/facebook/sam3)
|
| 27 |
+
for the Rust [`cortiq`](https://github.com/infosave2007/cmf) runtime.
|
| 28 |
+
|
| 29 |
+
SAM3 is Meta's Segment Anything Model 3: promptable instance segmentation
|
| 30 |
+
with a CLIP text encoder, a ViT trunk with windowed attention, a DETR-style
|
| 31 |
+
detector with 200 queries, a geometry encoder (boxes and points), a mask
|
| 32 |
+
decoder, and a video tracker (memory bank). This repository stores every
|
| 33 |
+
model section plus the vendor configuration and BPE tokenizer in one
|
| 34 |
+
`sam3-q8_2f.cmf` file: quantized q8_2f for 2-D projections, f16 for
|
| 35 |
+
convolutions, f32 for norms, geometry tables and the text tower.
|
| 36 |
+
|
| 37 |
+
Requires `cortiq` **v0.8.34** or newer (the release that ships
|
| 38 |
+
`sam3-pack`, the `segment` CLI and the video tracker).
|
| 39 |
+
|
| 40 |
+
[English](#files) · [Русский](#русский-перевод) · [中文](#中文说明)
|
| 41 |
+
|
| 42 |
+
## Files
|
| 43 |
+
|
| 44 |
+
| File | Size | SHA-256 |
|
| 45 |
+
|---|---|---|
|
| 46 |
+
| `sam3-q8_2f.cmf` | 1,962,174,217 bytes (1.96 GB) | `a7098fd061e0f884b40453fe684a4919113a2942ba9052c46fa208342fbcd3d1` |
|
| 47 |
+
| `LICENSE` | upstream Sam License | — |
|
| 48 |
+
| `SHA256SUMS.txt` | checksums | — |
|
| 49 |
+
|
| 50 |
+
`sam3-q8_2f.cmf` embeds, in one file: the vision trunk (ViT with windowed
|
| 51 |
+
attention), the SAM2-style FPN neck, the CLIP text tower (BPE tokenizer
|
| 52 |
+
included), the DETR-style detector, the geometry encoder, the mask decoder
|
| 53 |
+
and the full video tracker (`trk.*` — memory bank + its mask decoder), plus
|
| 54 |
+
the vendor configs.
|
| 55 |
+
|
| 56 |
+
## Usage
|
| 57 |
+
|
| 58 |
+
Build or install `cortiq` v0.8.34+, then:
|
| 59 |
+
|
| 60 |
+
```bash
|
| 61 |
+
# Text-prompt segmentation of one image -> mask overlay PNG + boxes/scores JSON
|
| 62 |
+
cortiq segment sam3-q8_2f.cmf image.png --prompt "red square"
|
| 63 |
+
|
| 64 |
+
# Video (or a directory of PNG frames) -> memory-bank tracker:
|
| 65 |
+
# a folder of overlay PNGs (mask + stable track id) and tracks.json
|
| 66 |
+
# with per-frame boxes, scores and id stability
|
| 67 |
+
cortiq segment sam3-q8_2f.cmf frames/ --prompt "red square" --out overlays/ --json tracks.json
|
| 68 |
+
cortiq segment sam3-q8_2f.cmf clip.mp4 --prompt "red square" --out overlays/ --json tracks.json
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
+
Video input accepts a PNG-frame directory or an mp4/mkv/webm/mov/avi file
|
| 72 |
+
(frames are extracted with `ffmpeg`). The tracker keeps one stable track id
|
| 73 |
+
across frames.
|
| 74 |
+
|
| 75 |
+
To re-pack the model from the upstream `facebook/sam3` export:
|
| 76 |
+
|
| 77 |
+
```bash
|
| 78 |
+
cortiq sam3-pack /path/to/sam3-export --quant q8_2f --out sam3-q8_2f.cmf
|
| 79 |
+
```
|
| 80 |
+
|
| 81 |
+
## Validation
|
| 82 |
+
|
| 83 |
+
- CPU parity against the HF reference (`transformers`): 6/6 stages pass
|
| 84 |
+
(text tower, trunk, encoder, decoder, boxes, masks).
|
| 85 |
+
- Video tracker parity: memory conditioning cos >= 0.9999 vs the HF
|
| 86 |
+
reference on a 3-frame fixture.
|
| 87 |
+
- End-to-end on a synthetic 8-frame video: the object is found in every
|
| 88 |
+
frame (box within <=1 px of ground truth), one stable track id across all
|
| 89 |
+
frames, mask IoU = 1.000 per frame.
|
| 90 |
+
|
| 91 |
+
## Русский-перевод
|
| 92 |
+
|
| 93 |
+
Нативная однофайловая CMF-упаковка
|
| 94 |
+
[facebook/sam3](https://huggingface.co/facebook/sam3) для Rust-рантайма
|
| 95 |
+
[`cortiq`](https://github.com/infosave2007/cmf).
|
| 96 |
+
|
| 97 |
+
Один файл `sam3-q8_2f.cmf` (1.96 GB) содержит: vision trunk (ViT с оконным
|
| 98 |
+
вниманием), FPN-шейку, CLIP text tower с BPE-токенизатором, DETR-детектор,
|
| 99 |
+
геометрический энкодер, mask decoder и полный видео-трекер (memory bank).
|
| 100 |
+
Квантование: q8_2f для 2-D проекций, f16 для свёрток, f32 для норм,
|
| 101 |
+
геометрических таблиц и текстовой башни.
|
| 102 |
+
|
| 103 |
+
Требуется `cortiq` **v0.8.34** или новее.
|
| 104 |
+
|
| 105 |
+
```bash
|
| 106 |
+
# Изображение
|
| 107 |
+
cortiq segment sam3-q8_2f.cmf image.png --prompt "красный квадрат"
|
| 108 |
+
|
| 109 |
+
# Видео (mp4/mkv/webm/папка PNG) — трекер с одним стабильным track id
|
| 110 |
+
cortiq segment sam3-q8_2f.cmf frames/ --prompt "красный квадрат" --out overlays/ --json tracks.json
|
| 111 |
+
```
|
| 112 |
+
|
| 113 |
+
Проверки: паритет CPU с эталоном HF — 6/6 стадий; паритет видео-трекера
|
| 114 |
+
cos >= 0.9999; сквозной тест на синтетическом 8-кадровом видео — объект
|
| 115 |
+
найден в каждом кадре (бокс <=1 px от эталона), один стабильный track id,
|
| 116 |
+
IoU маски 1.000.
|
| 117 |
+
|
| 118 |
+
## 中文说明
|
| 119 |
+
|
| 120 |
+
[facebook/sam3](https://huggingface.co/facebook/sam3) 的单文件 CMF 打包,
|
| 121 |
+
用于 Rust 运行时 [`cortiq`](https://github.com/infosave2007/cmf)。
|
| 122 |
+
|
| 123 |
+
单个 `sam3-q8_2f.cmf`(1.96 GB)包含视觉主干(ViT + 窗口注意力)、FPN、
|
| 124 |
+
CLIP 文本塔(含 BPE 分词器)、DETR 检测器、几何编码器、掩码解码器和完整
|
| 125 |
+
视频追踪器(记忆库)。量化:2-D 投影 q8_2f、卷积 f16、归一化/几何表/文本
|
| 126 |
+
塔 f32。需要 `cortiq` **v0.8.34** 或更新版本。
|
| 127 |
+
|
| 128 |
+
```bash
|
| 129 |
+
cortiq segment sam3-q8_2f.cmf image.png --prompt "红色方块"
|
| 130 |
+
cortiq segment sam3-q8_2f.cmf frames/ --prompt "红色方块" --out overlays/ --json tracks.json
|
| 131 |
+
```
|
| 132 |
+
|
| 133 |
+
## License
|
| 134 |
+
|
| 135 |
+
Weights are distributed under the upstream
|
| 136 |
+
[Sam License](https://huggingface.co/facebook/sam3/blob/main/LICENSE) from
|
| 137 |
+
`facebook/sam3`. This repository only re-packages them; see `LICENSE`.
|