phanerozoic commited on
Commit
70f9f7b
·
verified ·
1 Parent(s): 47a1863

Add RF100-VL cross-domain detection benchmark

Browse files
Files changed (1) hide show
  1. README.md +31 -0
README.md CHANGED
@@ -121,6 +121,37 @@ To test whether the detection head's features generalize beyond COCO, the shippi
121
 
122
  The detection head wins 17 of 20 domains, loses 3, with mean AR@100 +3.8 over the 5× larger FCOS baseline. The largest gaps are on domains far from COCO's distribution: aquarium-combined (+10.6), trail-camera (+9.5), new-defects-in-wood (+9.0), lacrosse-object-detection (+8.7), orionproducts (+8.5), the-dreidel-project (+7.4), soda-bottles (+6.3), all-elements (+5.6). The three losses are small (≤0.6 AR) on domains with very low absolute AR for both heads (gwhd2021 wheat-head crops, water-meter digit reads, x-ray-id anatomical landmarks). The interpretation is that the backbone's multi-teacher distilled features produce representations general enough that a frozen head one-fifth the FCOS size transfers across wildly different visual domains at the same level or better.
123
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
124
  ### Cross-Dataset Segmentation Transfer
125
 
126
  A separate BN+1×1 linear probe with the same training recipe as the ADE20K head, on the frozen backbone. The backbone was never exposed to driving scenes during EUPE distillation or Argus head training.
 
121
 
122
  The detection head wins 17 of 20 domains, loses 3, with mean AR@100 +3.8 over the 5× larger FCOS baseline. The largest gaps are on domains far from COCO's distribution: aquarium-combined (+10.6), trail-camera (+9.5), new-defects-in-wood (+9.0), lacrosse-object-detection (+8.7), orionproducts (+8.5), the-dreidel-project (+7.4), soda-bottles (+6.3), all-elements (+5.6). The three losses are small (≤0.6 AR) on domains with very low absolute AR for both heads (gwhd2021 wheat-head crops, water-meter digit reads, x-ray-id anatomical landmarks). The interpretation is that the backbone's multi-teacher distilled features produce representations general enough that a frozen head one-fifth the FCOS size transfers across wildly different visual domains at the same level or better.
123
 
124
+ ### Cross-domain detection benchmark (RF100-VL subset)
125
+
126
+ The same 20-domain RF100-VL class-agnostic AR@100 protocol applied to standard 2026 detector families: YOLO26 (n/s/m/l/x) from Ultralytics and RF-DETR (Nano/Small/Medium/Large) from Roboflow. All models trained on COCO; none saw RF100-VL. Each model evaluated at its native input resolution (YOLO26 variants at 768, RF-DETR at 384/512/576/704 for Nano/Small/Medium/Large, Argus at 768), max 100 detections per image, class labels ignored. Columns after Argus are sorted by mean AR@100 descending.
127
+
128
+ | domain | Argus+FCOS | Argus+(current) | RF-DETR-L | RF-DETR-M | RF-DETR-S | YOLO26l | RF-DETR-N | YOLO26x | YOLO26m | YOLO26s | YOLO26n |
129
+ |---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
130
+ | actions | 37.5 | 39.6 | 46.0 | 43.9 | 42.4 | 47.0 | 37.8 | 49.1 | 45.8 | 43.4 | 35.8 |
131
+ | aerial-airport | 16.1 | 17.3 | 19.5 | 16.4 | 15.5 | 13.9 | 13.1 | 17.8 | 14.1 | 10.0 | 8.9 |
132
+ | all-elements | 2.2 | 7.9 | 25.5 | 23.9 | 21.6 | 8.4 | 19.8 | 6.9 | 10.2 | 8.6 | 7.6 |
133
+ | aquarium-combined | 47.5 | 58.2 | 56.5 | 55.0 | 53.1 | 52.7 | 51.1 | 52.0 | 50.7 | 50.1 | 44.1 |
134
+ | defect-detection | 0.1 | 0.3 | 8.7 | 8.9 | 9.6 | 7.7 | 8.9 | 7.6 | 5.6 | 4.8 | 5.2 |
135
+ | dentalai | 0.2 | 0.9 | 11.5 | 11.6 | 11.2 | 8.0 | 8.9 | 3.6 | 2.0 | 5.0 | 9.8 |
136
+ | flir-camera-objects | 53.1 | 54.3 | 54.6 | 52.5 | 49.8 | 55.4 | 45.8 | 56.1 | 54.3 | 52.1 | 48.3 |
137
+ | gwhd2021 | 1.7 | 1.5 | 20.1 | 17.7 | 15.5 | 21.1 | 10.0 | 24.1 | 16.2 | 10.9 | 9.4 |
138
+ | lacrosse-object-detection | 57.9 | 66.6 | 58.5 | 57.7 | 57.7 | 59.7 | 56.0 | 60.7 | 60.2 | 59.9 | 58.9 |
139
+ | new-defects-in-wood | 5.6 | 14.6 | 24.0 | 22.8 | 21.5 | 17.7 | 20.6 | 16.9 | 15.4 | 17.0 | 19.4 |
140
+ | orionproducts | 17.1 | 25.5 | 15.8 | 14.5 | 14.8 | 30.9 | 13.2 | 11.6 | 26.5 | 29.8 | 22.1 |
141
+ | paper-parts | 19.3 | 22.2 | 25.9 | 23.7 | 25.4 | 5.7 | 32.5 | 13.2 | 8.1 | 13.5 | 11.4 |
142
+ | recode-waste | 11.4 | 11.8 | 40.2 | 36.5 | 34.0 | 38.4 | 28.3 | 32.2 | 36.3 | 36.6 | 40.4 |
143
+ | soda-bottles | 29.6 | 35.8 | 45.3 | 45.0 | 44.6 | 42.9 | 43.5 | 40.4 | 34.2 | 35.8 | 32.6 |
144
+ | the-dreidel-project | 57.7 | 65.1 | 64.7 | 66.1 | 63.1 | 58.8 | 59.4 | 62.4 | 58.3 | 53.4 | 53.7 |
145
+ | trail-camera | 60.1 | 69.6 | 67.4 | 67.2 | 67.0 | 64.1 | 66.0 | 64.8 | 61.7 | 58.5 | 55.4 |
146
+ | water-meter | 0.7 | 0.0 | 23.5 | 23.1 | 17.1 | 11.3 | 22.5 | 9.2 | 14.8 | 16.8 | 35.5 |
147
+ | wb-prova | 83.6 | 86.2 | 80.8 | 79.3 | 79.4 | 79.3 | 77.3 | 81.0 | 78.9 | 77.0 | 75.7 |
148
+ | wildfire-smoke | 0.3 | 0.5 | 0.3 | 0.6 | 2.0 | 0.5 | 0.9 | 0.5 | 2.0 | 0.5 | 0.8 |
149
+ | x-ray-id | 0.0 | 0.0 | 0.1 | 0.1 | 0.1 | 0.1 | 0.1 | 0.0 | 0.0 | 0.0 | 0.0 |
150
+ | **mean AR@100** | **25.1** | **28.9** | **34.4** | **33.3** | **32.3** | **31.2** | **30.8** | **30.5** | **29.8** | **29.2** | **28.8** |
151
+ | **total params** | 102.1M | 89.0M | 33.9M | 33.7M | 32.1M | 26.3M | 30.5M | 59.0M | 21.9M | 10.0M | 2.6M |
152
+
153
+ The Argus total-params entries (102.1M and 89.0M) include the 86M EUPE-ViT-B backbone shared across five task heads. YOLO26 and RF-DETR entries are standalone detection models with their own backbones.
154
+
155
  ### Cross-Dataset Segmentation Transfer
156
 
157
  A separate BN+1×1 linear probe with the same training recipe as the ADE20K head, on the frozen backbone. The backbone was never exposed to driving scenes during EUPE distillation or Argus head training.