Commit ·
d3fd01b
1
Parent(s): bef05b3
Upload 3 files (#2)
Browse files- Upload 3 files (065f8d616842d63259e36a9a04ea18e87a49791f)
- README.md +18 -0
- country_breakdown.json +32 -0
- domain_metadata.json +0 -0
README.md
CHANGED
|
@@ -109,6 +109,22 @@ Every model DetectionBench has trained and evaluated on Global Wheat Head Datase
|
|
| 109 |
| wheat_head | 67.1 | 27.11 |
|
| 110 |
---
|
| 111 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 112 |
## Evaluation Visualizations
|
| 113 |
|
| 114 |
This model was evaluated with [Supervision](https://github.com/roboflow/supervision)'s detection metrics, which report mAP/Precision/Recall directly but don't produce PR-curve, F1-curve, or confusion-matrix plot images the way Ultralytics' validator does. See the Performance table above for Precision/Recall/F1 and the per-class table above for the full per-class mAP breakdown.
|
|
@@ -178,6 +194,8 @@ detections = model.predict("image.jpg", threshold=0.25)
|
|
| 178 |
checkpoint_best_total.pth
|
| 179 |
metrics.csv
|
| 180 |
config.json
|
|
|
|
|
|
|
| 181 |
gwhd_rfdetr-medium_showcase.jpg
|
| 182 |
README.md
|
| 183 |
```
|
|
|
|
| 109 |
| wheat_head | 67.1 | 27.11 |
|
| 110 |
---
|
| 111 |
|
| 112 |
+
## Per-Country Performance
|
| 113 |
+
|
| 114 |
+
Domain shift can matter more than the aggregate score above for field deployment, so this evaluates the same test split broken down by the contributing country/institution, using per-image domain metadata (`domain_metadata.json` in this repository) compiled by this project for this stratified evaluation -- not a file shipped with the original GWHD release. Each row below is computed by re-running this exact model's evaluation restricted to that country's images only -- the same mAP definition as the aggregate number above (Ultralytics' `model.val()` for YOLO, Supervision's `MeanAveragePrecision` for RF-DETR), just on a filtered subset, not a separate metric implementation. One test image with no resolvable country in the source metadata (a documented upstream duplicate-filename quirk) is excluded from every row below.
|
| 115 |
+
|
| 116 |
+
| Country | mAP@50 | mAP@50-95 | Test Images |
|
| 117 |
+
| --------------------------- | --------------- | ----------------- | ------------------ |
|
| 118 |
+
| Australia | 57.87 | 22.49 | 281 |
|
| 119 |
+
| China | 88.71 | 43.81 | 200 |
|
| 120 |
+
| Japan | 71.1 | 38.82 | 60 |
|
| 121 |
+
| Mexico | 68.54 | 27.99 | 205 |
|
| 122 |
+
| Sudan | 74.03 | 34.04 | 30 |
|
| 123 |
+
| US | 64.97 | 23.76 | 605 |
|
| 124 |
+
See `country_breakdown.json` (results) and `domain_metadata.json` (the country/growth-stage mapping used to compute them) in this repository for the raw data behind this table.
|
| 125 |
+
|
| 126 |
+
---
|
| 127 |
+
|
| 128 |
## Evaluation Visualizations
|
| 129 |
|
| 130 |
This model was evaluated with [Supervision](https://github.com/roboflow/supervision)'s detection metrics, which report mAP/Precision/Recall directly but don't produce PR-curve, F1-curve, or confusion-matrix plot images the way Ultralytics' validator does. See the Performance table above for Precision/Recall/F1 and the per-class table above for the full per-class mAP breakdown.
|
|
|
|
| 194 |
checkpoint_best_total.pth
|
| 195 |
metrics.csv
|
| 196 |
config.json
|
| 197 |
+
country_breakdown.json
|
| 198 |
+
domain_metadata.json
|
| 199 |
gwhd_rfdetr-medium_showcase.jpg
|
| 200 |
README.md
|
| 201 |
```
|
country_breakdown.json
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"Australia": {
|
| 3 |
+
"mAP50": 0.5787455439567566,
|
| 4 |
+
"mAP50_95": 0.22489845752716064,
|
| 5 |
+
"num_images": 281
|
| 6 |
+
},
|
| 7 |
+
"China": {
|
| 8 |
+
"mAP50": 0.8871119022369385,
|
| 9 |
+
"mAP50_95": 0.4380679726600647,
|
| 10 |
+
"num_images": 200
|
| 11 |
+
},
|
| 12 |
+
"Japan": {
|
| 13 |
+
"mAP50": 0.7110053896903992,
|
| 14 |
+
"mAP50_95": 0.38823771476745605,
|
| 15 |
+
"num_images": 60
|
| 16 |
+
},
|
| 17 |
+
"Mexico": {
|
| 18 |
+
"mAP50": 0.6854299306869507,
|
| 19 |
+
"mAP50_95": 0.27985256910324097,
|
| 20 |
+
"num_images": 205
|
| 21 |
+
},
|
| 22 |
+
"Sudan": {
|
| 23 |
+
"mAP50": 0.7403203248977661,
|
| 24 |
+
"mAP50_95": 0.34035342931747437,
|
| 25 |
+
"num_images": 30
|
| 26 |
+
},
|
| 27 |
+
"US": {
|
| 28 |
+
"mAP50": 0.6496736407279968,
|
| 29 |
+
"mAP50_95": 0.23756475746631622,
|
| 30 |
+
"num_images": 605
|
| 31 |
+
}
|
| 32 |
+
}
|
domain_metadata.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|