mlboydaisuke commited on
Commit
42c5be7
·
verified ·
1 Parent(s): c7a8ef9

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +21 -8
README.md CHANGED
@@ -22,13 +22,23 @@ on the next call, with the same weights.
22
  -> scores (1,900,256), boxes (1,900,4)
23
  ```
24
 
25
- | build | file | MB | scores corr | boxes corr | Mac ms* |
26
- |---|---|---|---|---|---|
27
- | fp32 | `grounding_dino_tiny_xnnpack_fp32.pte` | 690.3 | 0.999833 | 0.998992 | 584.1 |
 
28
 
29
  \*Mac arm64, single process, median of 5 — a reference point for relative cost, not a
30
- device number. Torch eager fp32 on the same machine: 376.7 ms. XNNPACK delegate
31
- coverage 63.0% (3503/5559 ops).
 
 
 
 
 
 
 
 
 
32
 
33
  **There is no Core ML build.** The graph reshapes to rank 6 and Core ML's ceiling is 5 —
34
  the same wall RAFT-small hits with its correlation volume. This one is XNNPACK only, and
@@ -69,11 +79,14 @@ as the label.
69
  The number that decides is detections, not correlation. Against eager PyTorch on five
70
  street photographs at threshold 0.3:
71
 
72
- **26 of 26 of eager's detections are reproduced**, each at IoU > 0.5 with the same prompt
73
- token. That includes a photo with 17 detections in it.
 
 
 
74
 
75
  ```bash
76
- python convert/audit_int8.py grounding_dino_tiny --variant fp32
77
  python convert/verify_cards.py grounding_dino_tiny
78
  ```
79
 
 
22
  -> scores (1,900,256), boxes (1,900,4)
23
  ```
24
 
25
+ | build | file | MB | scores corr | boxes corr | detections agreeing | Mac ms* |
26
+ |---|---|---|---|---|---|---|
27
+ | fp32 | `grounding_dino_tiny_xnnpack_fp32.pte` | 690.3 | 0.999833 | 0.998992 | **26/26** | 584.1 |
28
+ | int8 (dynamic) | `grounding_dino_tiny_xnnpack_int8.pte` | **254.4** | 0.804530 | 0.777331 | **26/27** | 738.7 |
29
 
30
  \*Mac arm64, single process, median of 5 — a reference point for relative cost, not a
31
+ device number. Torch eager fp32 on the same machine: 376.7 ms. XNNPACK delegate coverage
32
+ 63.0% (3503/5559 ops) for fp32, 71.5% for int8.
33
+
34
+ **Read that int8 row carefully.** Its correlations are 0.80 and 0.78, which on any other
35
+ model on this shelf would withdraw the build. Here they are dominated by the queries that
36
+ detect nothing, whose coordinates are free to move without changing an answer — and the
37
+ detections themselves hold: all 26 that eager finds are found, at IoU > 0.5 with the same
38
+ prompt token, plus one box eager does not have. It is 37% of the fp32 file and it is what
39
+ you would put on a phone, but the evidence behind it is 26 detections over five
40
+ photographs, not a COCO run, and one of those 27 boxes already disagrees. It is also
41
+ slower than fp32 on this Mac (738.7 ms against 584.1).
42
 
43
  **There is no Core ML build.** The graph reshapes to rank 6 and Core ML's ceiling is 5 —
44
  the same wall RAFT-small hits with its correlation volume. This one is XNNPACK only, and
 
79
  The number that decides is detections, not correlation. Against eager PyTorch on five
80
  street photographs at threshold 0.3:
81
 
82
+ **fp32 reproduces 26 of 26**, each at IoU > 0.5 with the same prompt token, and invents
83
+ nothing. That includes a photo with 17 detections in it. **int8 scores 26 of 27** — it
84
+ finds all of eager's and one box besides. The denominator is whichever build found more,
85
+ so an invented detection costs the same as a missed one; counting only the reference's
86
+ detections would give a build that returns every box in the image a perfect score.
87
 
88
  ```bash
89
+ python convert/audit_int8.py grounding_dino_tiny --variant fp32 # or int8
90
  python convert/verify_cards.py grounding_dino_tiny
91
  ```
92