Improve model card: add metadata, paper/code/project links, and update content

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +31 -11
README.md CHANGED
@@ -1,43 +1,62 @@
1
- # FoMo4Wheat
2
- The official implementation of the paper **Crop-specific Vision Foundation Model enabling Generalized Field Monitoring**
 
 
3
 
4
- # Abstract
5
- Vision-driven in-field crop monitoring is essential for advancing digital agriculture whether supporting commercial decisions on-farm or augmenting research experiments in breeding and agronomy. Existing crop vision models struggle to generalize across fine-scale, highly variable canopy structures, and fluctuating outdoor environments. In this work, we present FoMo4Wheat, one of the first crop-orientated vision foundation models and demonstrate that delivers strong performance across a wide range of agricultural vision tasks. Centered on wheat, the most globally significant food crop, we curated ImAg4Wheat—the largest and most diverse wheat image dataset to date. It comprises 2.5 million high-resolution images collected over a decade from breeding and experimental fields, spanning more than 2,000 genotypes and 500 distinct environmental conditions across 30 global sites. A suite of FoMo4Wheat models was pre-trained using self-supervised learning on this dataset. Benchmark results across ten crop-related downstream tasks show that FoMo4Wheat consistently outperforms state-of-the-art models trained on general-domain datasets. Beyond strong cross-task generalization within wheat crops, FoMo4Wheat is highly robust in limited-data regimes but on previously unseen crop data. Notably, it contributes significantly to vision tasks in rice and multiplw crop/weed images, highlighting its cross-crop adaptability. In delivering one of the first open-source foundation models for wheat, our results demonstrate the value of such crop-specific foundation models that will support the development of versatile high-performing vision systems in crop breeding and precision agriculture. 
6
 
7
- # Installation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  The training and evaluation code is developed with PyTorch 2.5.1 and requires Linux environment with multiple third-party dependencies. To set up all required dependencies for training and evaluation, please follow the instructions below:
9
  ```
10
  conda env create -f conda.yaml
11
  conda activate FoMo4Wheat
12
  ```
13
 
14
- # Data Preparation
15
  ImAg4Wheat comprises 2,500,000 million images over 2,000 wheat genotypes cultivated under 500 distinct environmental conditions across 30 sites in 10 countries spanning a decade, covering the full crop growth cycle. [ImAg4Wheat](https://huggingface.co/datasets/PheniX-Lab/ImAg4Wheat)
16
  (Note: The complete dataset will be made publicly available after the peer-review process of the associated paper is completed.)
17
 
18
- # Pretrained models
19
  | model | # of params | download |
20
  | :---------------------:| -----------: |:--------------:|
21
  | ViT-B/14 | 86 M | [FoMo4Wheat_base.pth](https://huggingface.co/PheniX-Lab/FoMo4Wheat/blob/main/weight/FoMo4Wheat_base.pth) |
22
  | ViT-L/14 | 300 M | [FoMo4Wheat_large.pth](https://huggingface.co/PheniX-Lab/FoMo4Wheat/blob/main/weight/FoMo4Wheat_large.pth) |
23
  | ViT-G/14 | 1,100 M | [FoMo4Wheat_giant.pth](https://huggingface.co/PheniX-Lab/FoMo4Wheat/blob/main/weight/FoMo4Wheat_giant.pth) |
24
 
25
- # Training
26
  **Training FoMo4Wheat on ImAg4Wheat**
27
 
28
  Run FoMo4Wheat training on 6 A800-80GB nodes (48 GPUs) in a SLURM cluster environment with submitit:
29
  ```
30
- MKL_NUM_THREADS=8 OMP_NUM_THREADS=8 python FoMo4Wheat/run/train/
31
  --nodes 6 \
32
  --config-file FoMo4Wheat/configs/train/vitg_14_224.yaml \
33
  --output-dir <PATH/TO/OUTPUT/DIR> \
34
  train.dataset_path=TestDataset:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>
35
  ```
36
 
37
- # License
38
  FoMo4Wheat code and model weights are released under the MIT License. See LICENSE for additional details.
39
 
40
- # Citation
41
  If you use our project in your research or wish to refer to the results of the project, please use the following BibTeX entry.
42
 
43
  ```bibtex
@@ -48,3 +67,4 @@ If you use our project in your research or wish to refer to the results of the p
48
  year={2025}
49
  note={contact:Shouyang Liu (shouyang.liu@njau.edu.cn),Hao Lu (hlu@hust.edu.cn),Yanfeng Ding (dingyf@njau.edu.cn)}
50
  }
 
 
1
+ ---
2
+ license: mit
3
+ pipeline_tag: image-feature-extraction
4
+ ---
5
 
6
+ # FoMo4Wheat: Toward reliable crop vision foundation models with globally curated data
 
7
 
8
+ Paper: [https://huggingface.co/papers/2509.06907](https://huggingface.co/papers/2509.06907)
9
+ Project Page: https://fomo4wheat.phenix-lab.com/
10
+ Code: https://github.com/PheniX-Lab/FoMo4Wheat
11
+
12
+ ## Abstract
13
+ Vision-driven field monitoring is central to digital agriculture, yet models built on general-domain pretrained backbones often fail to generalize across tasks, owing to the interaction of fine, variable canopy structures with fluctuating field conditions. We present FoMo4Wheat, one of the first crop-domain vision foundation model pretrained with self-supervision on ImAg4Wheat, the largest and most diverse wheat image dataset to date (2.5 million high-resolution images collected over a decade at 30 global sites, spanning >2,000 genotypes and >500 environmental conditions). This wheat-specific pretraining yields representations that are robust for wheat and transferable to other crops and weeds. Across ten in-field vision tasks at canopy and organ levels, FoMo4Wheat models consistently outperform state-of-the-art models pretrained on general-domain dataset. These results demonstrate the value of crop-specific foundation models for reliable in-field perception and chart a path toward a universal crop foundation model with cross-species and cross-task capabilities. FoMo4Wheat models and the ImAg4Wheat dataset are publicly available online: this https URL and this https URL . The demonstration website is: this https URL .
14
+
15
+ ## Demo
16
+ The demonstration website for inferring embeddings is located at [Demo](https://fomo4wheat.phenix-lab.com/).
17
+
18
+ https://github.com/user-attachments/assets/2f2f21b4-4638-41c6-8bdf-37d8ad458eb6
19
+
20
+ 🎥 **Visualization of Unlabeled wheat features.**
21
+
22
+ ## Method
23
+ <img width="1267" height="1459" alt="Fig 1" src="https://github.com/user-attachments/assets/1d095d9b-2de4-4080-b68c-7da83f12edc1" />
24
+ <b>Fig 1.</b> Overview of ImAg4Wheat dataset and FoMo4Wheat model.
25
+
26
+ ## Installation
27
  The training and evaluation code is developed with PyTorch 2.5.1 and requires Linux environment with multiple third-party dependencies. To set up all required dependencies for training and evaluation, please follow the instructions below:
28
  ```
29
  conda env create -f conda.yaml
30
  conda activate FoMo4Wheat
31
  ```
32
 
33
+ ## Data Preparation
34
  ImAg4Wheat comprises 2,500,000 million images over 2,000 wheat genotypes cultivated under 500 distinct environmental conditions across 30 sites in 10 countries spanning a decade, covering the full crop growth cycle. [ImAg4Wheat](https://huggingface.co/datasets/PheniX-Lab/ImAg4Wheat)
35
  (Note: The complete dataset will be made publicly available after the peer-review process of the associated paper is completed.)
36
 
37
+ ## Pretrained models
38
  | model | # of params | download |
39
  | :---------------------:| -----------: |:--------------:|
40
  | ViT-B/14 | 86 M | [FoMo4Wheat_base.pth](https://huggingface.co/PheniX-Lab/FoMo4Wheat/blob/main/weight/FoMo4Wheat_base.pth) |
41
  | ViT-L/14 | 300 M | [FoMo4Wheat_large.pth](https://huggingface.co/PheniX-Lab/FoMo4Wheat/blob/main/weight/FoMo4Wheat_large.pth) |
42
  | ViT-G/14 | 1,100 M | [FoMo4Wheat_giant.pth](https://huggingface.co/PheniX-Lab/FoMo4Wheat/blob/main/weight/FoMo4Wheat_giant.pth) |
43
 
44
+ ## Training
45
  **Training FoMo4Wheat on ImAg4Wheat**
46
 
47
  Run FoMo4Wheat training on 6 A800-80GB nodes (48 GPUs) in a SLURM cluster environment with submitit:
48
  ```
49
+ MKL_NUM_THREADS=8 OMP_NUM_THREADS=8 python FoMo4Wheat/run/train/ \
50
  --nodes 6 \
51
  --config-file FoMo4Wheat/configs/train/vitg_14_224.yaml \
52
  --output-dir <PATH/TO/OUTPUT/DIR> \
53
  train.dataset_path=TestDataset:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>
54
  ```
55
 
56
+ ## License
57
  FoMo4Wheat code and model weights are released under the MIT License. See LICENSE for additional details.
58
 
59
+ ## Citation
60
  If you use our project in your research or wish to refer to the results of the project, please use the following BibTeX entry.
61
 
62
  ```bibtex
 
67
  year={2025}
68
  note={contact:Shouyang Liu (shouyang.liu@njau.edu.cn),Hao Lu (hlu@hust.edu.cn),Yanfeng Ding (dingyf@njau.edu.cn)}
69
  }
70
+ ```