MillionTrees fine-tuned ResNet-50 RetinaNet (v0.25)

#7
by bw4sz - opened
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ figures/E014_overlays_wd.png filter=lfs diff=lfs merge=lfs -text
37
+ figures/E014_per_source_f1_wd.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -4,8 +4,90 @@ language:
4
  - en
5
  tags:
6
  - deepforest
 
 
 
 
7
  ---
8
 
9
  ## Tree Crown Detection in RGB Airborne Imagery
10
 
11
- The model was initially described in [Remote Sensing](https://www.mdpi.com/2072-4292/11/11/1309) on a single site. The prebuilt model uses a semi-supervised approach in which millions of moderate quality annotations are generated using a LiDAR unsupervised tree detection algorithm, followed by hand-annotations of RGB imagery from select sites. Comparisons among geographic sites were added to [Ecological Informatics](https://www.sciencedirect.com/science/article/pii/S157495412030011X). The model was further improved, and the Python package was released in [Methods in Ecology and Evolution](https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13472).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  - en
5
  tags:
6
  - deepforest
7
+ - object-detection
8
+ - tree-crown-detection
9
+ library_name: deepforest
10
+ pipeline_tag: object-detection
11
  ---
12
 
13
  ## Tree Crown Detection in RGB Airborne Imagery
14
 
15
+ DeepForest's default tree crown detector: a RetinaNet with a ResNet-50 FPN backbone that predicts one
16
+ bounding box per tree crown in RGB aerial and drone imagery.
17
+
18
+ The model was initially described in [Remote Sensing](https://www.mdpi.com/2072-4292/11/11/1309) on a single site. Comparisons among geographic sites were added to [Ecological Informatics](https://www.sciencedirect.com/science/article/pii/S157495412030011X). The model was further improved, and the Python package was released in [Methods in Ecology and Evolution](https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13472).
19
+
20
+ ### This version: fine-tuned on MillionTrees
21
+
22
+ These weights are the earlier NEON tree model fine-tuned on the [MillionTrees](https://milliontrees.idtrees.org)
23
+ TreeBoxes v0.25 within-distribution train split: about 13,600 images and 796,000 boxes from 15 sources
24
+ across many biomes, drone and airborne platforms, and resolutions. The checkpoint was selected on box F1
25
+ on the held-out validation split.
26
+
27
+ The earlier NEON-only weights are still in this repository's history; load them with `revision=` set
28
+ to the commit before this one.
29
+
30
+ ### Usage
31
+
32
+ ```python
33
+ from deepforest import main
34
+
35
+ model = main.deepforest(config_args={"nms_thresh": 0.2, "score_thresh": 0.2})
36
+ model.load_model("weecology/deepforest-tree")
37
+ boxes = model.predict_tile("image.tif", patch_size=400, patch_overlap=0.1)
38
+ ```
39
+
40
+ DeepForest's config sets `nms_thresh` and `score_thresh` when the model loads. This model was evaluated at
41
+ **NMS 0.2, score 0.2**; at the package default of NMS 0.05, recall drops noticeably on dense canopies.
42
+
43
+ ### Performance (MillionTrees TreeBoxes v0.25 test)
44
+
45
+ Predictions are matched to ground truth at IoU 0.4. AP40 is scored with up to 1000 detections per image.
46
+
47
+ | Split | Model | Recall | Mask-aware precision | F1 | AP40 | AP60 |
48
+ |---|---|---|---|---|---|---|
49
+ | Within-distribution | previous `deepforest-tree` (NEON) | 0.415 | 0.639 | 0.503 | 0.260 | 0.091 |
50
+ | Within-distribution | **this model** | 0.725 | 0.814 | 0.767 | 0.562 | 0.445 |
51
+ | Out-of-distribution* | previous `deepforest-tree` (NEON) | 0.540 | 0.781 | 0.638 | 0.298 | 0.123 |
52
+ | Out-of-distribution* | same recipe, OOD-trained | 0.738 | 0.913 | 0.816 | 0.520 | 0.354 |
53
+
54
+ \*The out-of-distribution row is a separate checkpoint trained with the same recipe but with whole sources
55
+ held out. It shows how the recipe does on sources it never saw. These weights were trained on the
56
+ within-distribution split.
57
+
58
+ ### Figures
59
+
60
+ ![F1 and AP40 for the previous release, the ResNet-50 fine-tune and the DINOv3 fine-tune](figures/E014_macro.png)
61
+
62
+ Macro-averaged over sources on the MillionTrees TreeBoxes v0.25 test splits. The out-of-distribution bars
63
+ come from checkpoints trained with whole sources held out.
64
+
65
+ ![Per-source F1 on the within-distribution test split](figures/E014_per_source_f1_wd.png)
66
+
67
+ Per source on the within-distribution test split. The new models improve on the NEON release everywhere
68
+ except NEON MultiTemporal, where all three are close. The largest gains are on sources unlike NEON:
69
+ drone imagery, palms, tropical forest and urban trees.
70
+
71
+ ![Predictions on held-out test tiles](figures/E014_overlays_wd.png)
72
+
73
+ The same held-out test tile for each model; purple is ground truth and orange is prediction. Rows are the
74
+ six largest test sources, each showing the image with that source's median number of trees, so the tiles
75
+ were not picked for gain. Some sources label only some of the trees (for example OAM-TCD and the palm
76
+ counting set), so an unmatched orange box is not always a false positive.
77
+
78
+ For a stronger but heavier model, see `weecology/deepforest-tree-dinov3` (RetinaNet with a DINOv3 ViT-L
79
+ backbone, which needs access to Meta's DINOv3 weights).
80
+
81
+ ### Citation
82
+
83
+ ```bibtex
84
+ @article{weinstein2020deepforest,
85
+ title={DeepForest: A Python package for RGB deep learning tree crown delineation},
86
+ author={Weinstein, Ben G and Marconi, Sergio and Aubry-Kientz, M{\'e}laine and Vincent, Gregoire and Senyondo, Henry and White, Ethan P},
87
+ journal={Methods in Ecology and Evolution},
88
+ volume={11},
89
+ number={12},
90
+ pages={1743--1751},
91
+ year={2020}
92
+ }
93
+ ```
config.json CHANGED
@@ -1,12 +1,10 @@
1
  {
2
- "config_args": null,
3
- "config_file": "deepforest_config.yml",
4
- "existing_train_dataloader": null,
5
- "existing_val_dataloader": null,
6
  "label_dict": {
7
  "Tree": 0
8
  },
9
- "model": null,
10
  "num_classes": 1,
11
- "transforms": null
 
12
  }
 
1
  {
2
+ "detections_per_img": 300,
 
 
 
3
  "label_dict": {
4
  "Tree": 0
5
  },
6
+ "nms_thresh": 0.2,
7
  "num_classes": 1,
8
+ "score_thresh": 0.2,
9
+ "topk_candidates": 1000
10
  }
figures/E014_macro.png ADDED
figures/E014_overlays_wd.png ADDED

Git LFS Details

  • SHA256: 62ba7d96a278d81ca860aa27c10277192e6fa4871b4a093690062ff7628e0714
  • Pointer size: 132 Bytes
  • Size of remote file: 3.6 MB
figures/E014_per_source_f1_wd.png ADDED

Git LFS Details

  • SHA256: 181464ecfb40f0ed30feed552afd54c5ef3aae7507e2738ba2cb6fdcdbc8efab
  • Pointer size: 131 Bytes
  • Size of remote file: 188 kB
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d37a7af0b5ba2754282a80d41b4c7b66e8c7149234df5bf3f41bdaf57d329fc8
3
- size 129049980
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e755847e1162ca312013a7cf8f7f6f4d13e1a63e117a0f02f8c7cf9f414ebba9
3
+ size 129048172