Abhinav Bhatia commited on
Commit
3a3dac4
·
verified ·
1 Parent(s): fd83431

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +108 -0
README.md ADDED
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-sa-4.0
3
+ tags:
4
+ - image-classification
5
+ - agriculture
6
+ - plant-disease
7
+ - onnx
8
+ datasets:
9
+ - mohanty/PlantVillage
10
+ metrics:
11
+ - accuracy
12
+ - f1
13
+ ---
14
+
15
+ # CropGuard - Crop Disease Classifier (38 classes)
16
+
17
+ ResNet50 fine-tuned on PlantVillage, exported to ONNX and dynamically quantised to INT8 for
18
+ CPU-only serving. Part of [CropGuard](https://github.com/abhinav7289A/CropGuard), an
19
+ end-to-end MLOps pipeline.
20
+
21
+ ## Results (held-out test set, n=8,125)
22
+
23
+ | Model | Accuracy | Macro-F1 | Note |
24
+ |---|---|---|---|
25
+ | fp32 | 0.9911 | 0.9865 | reference |
26
+ | INT8 (dynamic) | not evaluated | not evaluated | not served - see below |
27
+
28
+ Macro-F1 is the metric to read here, not accuracy: the dataset is imbalanced ~36x, so accuracy
29
+ is dominated by the largest classes.
30
+
31
+ **Only `cropguard.onnx` (fp32) is published.** Dynamic INT8 quantization was tried and is
32
+ *not* shipped: `quantize_dynamic` rewrites every `Conv` into `ConvInteger`, which ONNX
33
+ Runtime's CPU backend has no optimized kernel for. Measured on an Intel Alder Lake CPU it ran
34
+ at **1567 ms/image against 19 ms/image for fp32** - a 75x regression in exchange for 4x less
35
+ disk. Dynamic quantization suits MatMul-dominated models (Transformers, RNNs), not CNNs; the
36
+ correct approach for a Conv-heavy network is *static* quantization with a calibration set,
37
+ which emits the optimized `QLinearConv`. That is not built yet.
38
+
39
+ ## The split is grouped by leaf, and that matters
40
+
41
+ PlantVillage contains 54,305 images of only ~7,600 *distinct physical leaves* -
42
+ roughly 7 photographs of each. A standard per-image stratified split scatters those
43
+ near-duplicates across train and test, so a model can score well by memorising leaf identity
44
+ rather than learning disease morphology. Measured on this dataset, a naive split leaves
45
+ **74.2% of test images sharing a leaf with training**.
46
+
47
+ This model was trained on a **leaf-grouped** split instead:
48
+
49
+ | | Naive stratified | Grouped (used here) |
50
+ |---|---|---|
51
+ | test images sharing a leaf with train | 74.2% | **0.0%** |
52
+ | train / val / test | - | 38,008 / 8,172 / 8,125 |
53
+
54
+ So the accuracy above is measured on a holdout with no leaf overlap. Note that it is *not*
55
+ much lower than typically published PlantVillage figures - the honest reading is that this
56
+ dataset is genuinely easy, not that leakage was inflating everything.
57
+
58
+ ## Intended use
59
+
60
+ Identifying disease on **single leaves photographed against a plain background**, matching the
61
+ PlantVillage capture protocol.
62
+
63
+ ## Limitations - read before deploying this
64
+
65
+ - **Lab images, not field images.** Every training image is a detached leaf on a uniform
66
+ background under controlled lighting. Real photographs from a farm - variable lighting,
67
+ occlusion, multiple leaves, soil backgrounds - are a different distribution, and published
68
+ work on this dataset reports large drops there. This model has **not** been evaluated on
69
+ field photographs.
70
+ - **Per-class metrics for rare classes are noisy.** Eight classes have fewer than 100 test
71
+ images; the smallest (`Potato___healthy`) has 24. A recall of 0.833 there is 4 mistakes, and
72
+ its confidence interval spans roughly +/-15 points. Do not read those per-class numbers as
73
+ precise.
74
+ - **Confidence is not calibrated.** Training used label smoothing (0.1), which deliberately
75
+ caps confidence, so predicted probabilities are expected to understate. No temperature
76
+ scaling has been fitted.
77
+ - **`uncertainty` is predictive entropy**, not epistemic uncertainty. It cannot distinguish an
78
+ ambiguous input from one far outside the training distribution - an out-of-distribution
79
+ image can produce confidently wrong output with low entropy.
80
+ - 38 classes across 14 crops only. Anything outside that set is silently forced into one of
81
+ them.
82
+
83
+ ## Training
84
+
85
+ ResNet50 (timm, ImageNet-pretrained), 224x224, batch 64, AdamW (lr 3e-4, weight decay 1e-4),
86
+ cosine schedule, label smoothing 0.1, medium augmentation, 12 epochs, mixed precision.
87
+ Checkpoint selected on `val_f1_macro`.
88
+
89
+ ## Usage
90
+
91
+ ```python
92
+ import numpy as np, onnxruntime as ort
93
+ from huggingface_hub import hf_hub_download
94
+
95
+ path = hf_hub_download("XiElonMAsk/cropguard-models", "cropguard.onnx")
96
+ session = ort.InferenceSession(path, providers=["CPUExecutionProvider"])
97
+
98
+ # Preprocessing must match training: resize short side to 256, centre-crop 224,
99
+ # scale to [0,1], normalise with ImageNet mean/std, NCHW.
100
+ logits = session.run(["logits"], {"input": batch})[0]
101
+ ```
102
+
103
+ `cropguard.serving.model_loader` in the repo implements exactly that preprocessing.
104
+
105
+ ## Citation
106
+
107
+ Dataset: Mohanty, Hughes & Salathe (2016), *Using deep learning for image-based plant disease
108
+ detection*, Frontiers in Plant Science.