mlboydaisuke commited on
Commit
5b63d11
·
verified ·
1 Parent(s): 031d684

RF-DETR nano/small/medium/large fp32 .aimodel + card + demos

Browse files
.gitattributes CHANGED
@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ demo_coco_252219.jpg filter=lfs diff=lfs merge=lfs -text
37
+ demo_coco_cats.jpg filter=lfs diff=lfs merge=lfs -text
38
+ rfdetr-large_float32.aimodel/main.mlirb filter=lfs diff=lfs merge=lfs -text
39
+ rfdetr-medium_float32.aimodel/main.mlirb filter=lfs diff=lfs merge=lfs -text
40
+ rfdetr-nano_float32.aimodel/main.mlirb filter=lfs diff=lfs merge=lfs -text
41
+ rfdetr-small_float32.aimodel/main.mlirb filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: roboflow/rf-detr
4
+ tags:
5
+ - coreai
6
+ - aimodel
7
+ - object-detection
8
+ - rf-detr
9
+ - detr
10
+ - apple
11
+ - ios
12
+ - macos
13
+ pipeline_tag: object-detection
14
+ ---
15
+
16
+ # RF-DETR — Core AI (`.aimodel`)
17
+
18
+ [RF-DETR](https://github.com/roboflow/rf-detr) (Roboflow's real-time detection
19
+ transformer, COCO-pretrained) converted to Apple **Core AI** for iOS 27 / macOS 27 —
20
+ the answer to [apple/coreai-models#14](https://github.com/apple/coreai-models/issues/14).
21
+ **DETR family = no NMS**: post-processing is one sigmoid + top-k.
22
+
23
+ <p align="center"><img src="demo_coco_cats.jpg" width="440" alt="RF-DETR medium on Core AI"></p>
24
+
25
+ ## Files
26
+
27
+ | file | input | params | M4 Max GPU | iPhone 17 Pro GPU |
28
+ |---|---|---|---|---|
29
+ | `rfdetr-nano_float32.aimodel` | 384×384 | 30.5M | **8.6 ms** (~116 FPS) | **27.5 ms (~36 FPS live)** |
30
+ | `rfdetr-small_float32.aimodel` | 512×512 | 32.1M | **12.0 ms** (~83 FPS) | — |
31
+ | `rfdetr-medium_float32.aimodel` | 576×576 | 33.7M | **14.8 ms** (~68 FPS) | **56 ms (~17 FPS live)** |
32
+ | `rfdetr-large_float32.aimodel` | 704×704 | 33.9M | **19.1 ms** (~52 FPS) | — |
33
+
34
+ iPhone numbers are sustained live-camera medians from the
35
+ [CoreAIKit DetectCamera example](https://github.com/john-rocky/coreai-kit)
36
+ (`CameraFeed` + `ObjectDetector`, Release build).
37
+
38
+ fp32 is the ship dtype: it gates **detection-set exact** vs the PyTorch fp32 reference on
39
+ CPU and GPU (per confident detection: same class, IoU ≥ 0.999 measured, score within 2e-3),
40
+ and fp16 only bought ~7% latency on M4 Max while adding near-tie ranking noise.
41
+
42
+ ## Graph contract
43
+
44
+ ```
45
+ input "image" [1, 3, R, R] float32, RGB in [0, 1] (ImageNet mean/std folded in-graph)
46
+ output "dets" [1, 300, 4] boxes, cxcywh normalized to [0, 1]
47
+ output "labels" [1, 300, 91] raw class logits; column index = ORIGINAL COCO id (0 unused, 1=person … 17=cat … 90)
48
+ ```
49
+
50
+ Python decode sketch (Swift is the same three steps):
51
+
52
+ ```python
53
+ import numpy as np, coreai.runtime as rt
54
+
55
+ model = await rt.AIModel.load(path, rt.SpecializationOptions.default())
56
+ fn = model.load_function("main")
57
+ out = await fn({"image": rt.NDArray(rgb01)}) # rgb01: [1,3,R,R] in [0,1]
58
+ prob = 1 / (1 + np.exp(-out["labels"].numpy()[0])) # [300, 91]
59
+ scores, classes = prob.max(-1), prob.argmax(-1) # column index IS the COCO id
60
+ boxes = out["dets"].numpy()[0] # cxcywh, multiply by image W/H
61
+ keep = scores > 0.5 # done — no NMS
62
+ ```
63
+
64
+ ## Conversion
65
+
66
+ Exported with
67
+ [`conversion/export_rf_detr.py`](https://github.com/john-rocky/coreai-model-zoo/blob/main/conversion/export_rf_detr.py)
68
+ from `rfdetr==1.7.1` weights. The port surfaced four Core AI converter/runtime bugs
69
+ (float-arg `arange` abort, int64-comparison buffer clobber, GPU-delegate
70
+ floor/trunc/ceil = identity, cast-pair cancellation) — each worked around numerically
71
+ identically; details and minimal repros in
72
+ [zoo/rf-detr.md](https://github.com/john-rocky/coreai-model-zoo/blob/main/zoo/rf-detr.md).
73
+
74
+ License: Apache-2.0 (upstream RF-DETR code and COCO-pretrained weights are Apache-2.0).
demo_coco_252219.jpg ADDED

Git LFS Details

  • SHA256: 5712cf0fd3e1e46ae9670c3d04a6159c02f833b774a9e31beff6394a1c028dce
  • Pointer size: 131 Bytes
  • Size of remote file: 122 kB
demo_coco_cats.jpg ADDED

Git LFS Details

  • SHA256: 44f0219c22d04b15145544ad463b9e658bf9b5fd37c77d3d305d077efdccf974
  • Pointer size: 131 Bytes
  • Size of remote file: 139 kB
rfdetr-large_float32.aimodel/main.hash ADDED
@@ -0,0 +1 @@
 
 
1
+ �Q>�����76&�n�0���za���ogF
rfdetr-large_float32.aimodel/main.mlirb ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f2513ed3e3fcf3aaee863736268c6ef930bfb5e3ba017a619b98856f67160446
3
+ size 121906905
rfdetr-large_float32.aimodel/metadata.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "assetVersion" : "2.0"
3
+ }
rfdetr-medium_float32.aimodel/main.hash ADDED
@@ -0,0 +1 @@
 
 
1
+ ��{oW\J�2d׋���xReD��=�߮�
rfdetr-medium_float32.aimodel/main.mlirb ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:03f997017b6f575c4ac53264d78bfe168eb078520865441ed5c43da404dfaef2
3
+ size 120911038
rfdetr-medium_float32.aimodel/metadata.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "assetVersion" : "2.0"
3
+ }
rfdetr-nano_float32.aimodel/main.hash ADDED
@@ -0,0 +1 @@
 
 
1
+ ]�Gέ*�O_�3S��
rfdetr-nano_float32.aimodel/main.mlirb ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d8747cead2af04f5f9a3353f2a0a3fc0d5830f3979aa12a781c870c287e3e9b
3
+ size 107962969
rfdetr-nano_float32.aimodel/metadata.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "assetVersion" : "2.0"
3
+ }
rfdetr-small_float32.aimodel/main.hash ADDED
@@ -0,0 +1 @@
 
 
1
+ �� ���Lb��=�p<��gֲ��|�(P���
rfdetr-small_float32.aimodel/main.mlirb ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e7d90cfecadd4c1e6216fba23df501703cd7d067d6b20495937ce028509fc1e5
3
+ size 114573344
rfdetr-small_float32.aimodel/metadata.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "assetVersion" : "2.0"
3
+ }