dronefreak commited on
Commit
d3fd01b
·
1 Parent(s): bef05b3

Upload 3 files (#2)

Browse files

- Upload 3 files (065f8d616842d63259e36a9a04ea18e87a49791f)

Files changed (3) hide show
  1. README.md +18 -0
  2. country_breakdown.json +32 -0
  3. domain_metadata.json +0 -0
README.md CHANGED
@@ -109,6 +109,22 @@ Every model DetectionBench has trained and evaluated on Global Wheat Head Datase
109
  | wheat_head | 67.1 | 27.11 |
110
  ---
111
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
112
  ## Evaluation Visualizations
113
 
114
  This model was evaluated with [Supervision](https://github.com/roboflow/supervision)'s detection metrics, which report mAP/Precision/Recall directly but don't produce PR-curve, F1-curve, or confusion-matrix plot images the way Ultralytics' validator does. See the Performance table above for Precision/Recall/F1 and the per-class table above for the full per-class mAP breakdown.
@@ -178,6 +194,8 @@ detections = model.predict("image.jpg", threshold=0.25)
178
  checkpoint_best_total.pth
179
  metrics.csv
180
  config.json
 
 
181
  gwhd_rfdetr-medium_showcase.jpg
182
  README.md
183
  ```
 
109
  | wheat_head | 67.1 | 27.11 |
110
  ---
111
 
112
+ ## Per-Country Performance
113
+
114
+ Domain shift can matter more than the aggregate score above for field deployment, so this evaluates the same test split broken down by the contributing country/institution, using per-image domain metadata (`domain_metadata.json` in this repository) compiled by this project for this stratified evaluation -- not a file shipped with the original GWHD release. Each row below is computed by re-running this exact model's evaluation restricted to that country's images only -- the same mAP definition as the aggregate number above (Ultralytics' `model.val()` for YOLO, Supervision's `MeanAveragePrecision` for RF-DETR), just on a filtered subset, not a separate metric implementation. One test image with no resolvable country in the source metadata (a documented upstream duplicate-filename quirk) is excluded from every row below.
115
+
116
+ | Country | mAP@50 | mAP@50-95 | Test Images |
117
+ | --------------------------- | --------------- | ----------------- | ------------------ |
118
+ | Australia | 57.87 | 22.49 | 281 |
119
+ | China | 88.71 | 43.81 | 200 |
120
+ | Japan | 71.1 | 38.82 | 60 |
121
+ | Mexico | 68.54 | 27.99 | 205 |
122
+ | Sudan | 74.03 | 34.04 | 30 |
123
+ | US | 64.97 | 23.76 | 605 |
124
+ See `country_breakdown.json` (results) and `domain_metadata.json` (the country/growth-stage mapping used to compute them) in this repository for the raw data behind this table.
125
+
126
+ ---
127
+
128
  ## Evaluation Visualizations
129
 
130
  This model was evaluated with [Supervision](https://github.com/roboflow/supervision)'s detection metrics, which report mAP/Precision/Recall directly but don't produce PR-curve, F1-curve, or confusion-matrix plot images the way Ultralytics' validator does. See the Performance table above for Precision/Recall/F1 and the per-class table above for the full per-class mAP breakdown.
 
194
  checkpoint_best_total.pth
195
  metrics.csv
196
  config.json
197
+ country_breakdown.json
198
+ domain_metadata.json
199
  gwhd_rfdetr-medium_showcase.jpg
200
  README.md
201
  ```
country_breakdown.json ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "Australia": {
3
+ "mAP50": 0.5787455439567566,
4
+ "mAP50_95": 0.22489845752716064,
5
+ "num_images": 281
6
+ },
7
+ "China": {
8
+ "mAP50": 0.8871119022369385,
9
+ "mAP50_95": 0.4380679726600647,
10
+ "num_images": 200
11
+ },
12
+ "Japan": {
13
+ "mAP50": 0.7110053896903992,
14
+ "mAP50_95": 0.38823771476745605,
15
+ "num_images": 60
16
+ },
17
+ "Mexico": {
18
+ "mAP50": 0.6854299306869507,
19
+ "mAP50_95": 0.27985256910324097,
20
+ "num_images": 205
21
+ },
22
+ "Sudan": {
23
+ "mAP50": 0.7403203248977661,
24
+ "mAP50_95": 0.34035342931747437,
25
+ "num_images": 30
26
+ },
27
+ "US": {
28
+ "mAP50": 0.6496736407279968,
29
+ "mAP50_95": 0.23756475746631622,
30
+ "num_images": 605
31
+ }
32
+ }
domain_metadata.json ADDED
The diff for this file is too large to render. See raw diff