infosave commited on
Commit
8875efc
·
verified ·
1 Parent(s): eb4e965

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +137 -0
README.md ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: sam-license
4
+ library_name: cortiq
5
+ base_model:
6
+ - facebook/sam3
7
+ base_model_relation: quantized
8
+ pipeline_tag: image-segmentation
9
+ tags:
10
+ - cmf
11
+ - cortiq
12
+ - image-segmentation
13
+ - sam3
14
+ - video-tracking
15
+ - q8_2f
16
+ - rust
17
+ language:
18
+ - en
19
+ - ru
20
+ - zh
21
+ ---
22
+
23
+ # SAM3 CMF
24
+
25
+ Native, single-file CMF packaging of
26
+ [facebook/sam3](https://huggingface.co/facebook/sam3)
27
+ for the Rust [`cortiq`](https://github.com/infosave2007/cmf) runtime.
28
+
29
+ SAM3 is Meta's Segment Anything Model 3: promptable instance segmentation
30
+ with a CLIP text encoder, a ViT trunk with windowed attention, a DETR-style
31
+ detector with 200 queries, a geometry encoder (boxes and points), a mask
32
+ decoder, and a video tracker (memory bank). This repository stores every
33
+ model section plus the vendor configuration and BPE tokenizer in one
34
+ `sam3-q8_2f.cmf` file: quantized q8_2f for 2-D projections, f16 for
35
+ convolutions, f32 for norms, geometry tables and the text tower.
36
+
37
+ Requires `cortiq` **v0.8.34** or newer (the release that ships
38
+ `sam3-pack`, the `segment` CLI and the video tracker).
39
+
40
+ [English](#files) · [Русский](#русский-перевод) · [中文](#中文说明)
41
+
42
+ ## Files
43
+
44
+ | File | Size | SHA-256 |
45
+ |---|---|---|
46
+ | `sam3-q8_2f.cmf` | 1,962,174,217 bytes (1.96 GB) | `a7098fd061e0f884b40453fe684a4919113a2942ba9052c46fa208342fbcd3d1` |
47
+ | `LICENSE` | upstream Sam License | — |
48
+ | `SHA256SUMS.txt` | checksums | — |
49
+
50
+ `sam3-q8_2f.cmf` embeds, in one file: the vision trunk (ViT with windowed
51
+ attention), the SAM2-style FPN neck, the CLIP text tower (BPE tokenizer
52
+ included), the DETR-style detector, the geometry encoder, the mask decoder
53
+ and the full video tracker (`trk.*` — memory bank + its mask decoder), plus
54
+ the vendor configs.
55
+
56
+ ## Usage
57
+
58
+ Build or install `cortiq` v0.8.34+, then:
59
+
60
+ ```bash
61
+ # Text-prompt segmentation of one image -> mask overlay PNG + boxes/scores JSON
62
+ cortiq segment sam3-q8_2f.cmf image.png --prompt "red square"
63
+
64
+ # Video (or a directory of PNG frames) -> memory-bank tracker:
65
+ # a folder of overlay PNGs (mask + stable track id) and tracks.json
66
+ # with per-frame boxes, scores and id stability
67
+ cortiq segment sam3-q8_2f.cmf frames/ --prompt "red square" --out overlays/ --json tracks.json
68
+ cortiq segment sam3-q8_2f.cmf clip.mp4 --prompt "red square" --out overlays/ --json tracks.json
69
+ ```
70
+
71
+ Video input accepts a PNG-frame directory or an mp4/mkv/webm/mov/avi file
72
+ (frames are extracted with `ffmpeg`). The tracker keeps one stable track id
73
+ across frames.
74
+
75
+ To re-pack the model from the upstream `facebook/sam3` export:
76
+
77
+ ```bash
78
+ cortiq sam3-pack /path/to/sam3-export --quant q8_2f --out sam3-q8_2f.cmf
79
+ ```
80
+
81
+ ## Validation
82
+
83
+ - CPU parity against the HF reference (`transformers`): 6/6 stages pass
84
+ (text tower, trunk, encoder, decoder, boxes, masks).
85
+ - Video tracker parity: memory conditioning cos >= 0.9999 vs the HF
86
+ reference on a 3-frame fixture.
87
+ - End-to-end on a synthetic 8-frame video: the object is found in every
88
+ frame (box within <=1 px of ground truth), one stable track id across all
89
+ frames, mask IoU = 1.000 per frame.
90
+
91
+ ## Русский-перевод
92
+
93
+ Нативная однофайловая CMF-упаковка
94
+ [facebook/sam3](https://huggingface.co/facebook/sam3) для Rust-рантайма
95
+ [`cortiq`](https://github.com/infosave2007/cmf).
96
+
97
+ Один файл `sam3-q8_2f.cmf` (1.96 GB) содержит: vision trunk (ViT с оконным
98
+ вниманием), FPN-шейку, CLIP text tower с BPE-токенизатором, DETR-детектор,
99
+ геометрический энкодер, mask decoder и полный видео-трекер (memory bank).
100
+ Квантование: q8_2f для 2-D проекций, f16 для свёрток, f32 для норм,
101
+ геометрических таблиц и текстовой башни.
102
+
103
+ Требуется `cortiq` **v0.8.34** или новее.
104
+
105
+ ```bash
106
+ # Изображение
107
+ cortiq segment sam3-q8_2f.cmf image.png --prompt "красный квадрат"
108
+
109
+ # Видео (mp4/mkv/webm/папка PNG) — трекер с одним стабильным track id
110
+ cortiq segment sam3-q8_2f.cmf frames/ --prompt "красный квадрат" --out overlays/ --json tracks.json
111
+ ```
112
+
113
+ Проверки: паритет CPU с эталоном HF — 6/6 стадий; паритет видео-трекера
114
+ cos >= 0.9999; сквозной тест на синтетическом 8-кадровом видео — объект
115
+ найден в каждом кадре (бокс <=1 px от эталона), один стабильный track id,
116
+ IoU маски 1.000.
117
+
118
+ ## 中文说明
119
+
120
+ [facebook/sam3](https://huggingface.co/facebook/sam3) 的单文件 CMF 打包,
121
+ 用于 Rust 运行时 [`cortiq`](https://github.com/infosave2007/cmf)。
122
+
123
+ 单个 `sam3-q8_2f.cmf`(1.96 GB)包含视觉主干(ViT + 窗口注意力)、FPN、
124
+ CLIP 文本塔(含 BPE 分词器)、DETR 检测器、几何编码器、掩码解码器和完整
125
+ 视频追踪器(记忆库)。量化:2-D 投影 q8_2f、卷积 f16、归一化/几何表/文本
126
+ 塔 f32。需要 `cortiq` **v0.8.34** 或更新版本。
127
+
128
+ ```bash
129
+ cortiq segment sam3-q8_2f.cmf image.png --prompt "红色方块"
130
+ cortiq segment sam3-q8_2f.cmf frames/ --prompt "红色方块" --out overlays/ --json tracks.json
131
+ ```
132
+
133
+ ## License
134
+
135
+ Weights are distributed under the upstream
136
+ [Sam License](https://huggingface.co/facebook/sam3/blob/main/LICENSE) from
137
+ `facebook/sam3`. This repository only re-packages them; see `LICENSE`.