Abhinav Bhatia commited on
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-sa-4.0
|
| 3 |
+
tags:
|
| 4 |
+
- image-classification
|
| 5 |
+
- agriculture
|
| 6 |
+
- plant-disease
|
| 7 |
+
- onnx
|
| 8 |
+
datasets:
|
| 9 |
+
- mohanty/PlantVillage
|
| 10 |
+
metrics:
|
| 11 |
+
- accuracy
|
| 12 |
+
- f1
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# CropGuard - Crop Disease Classifier (38 classes)
|
| 16 |
+
|
| 17 |
+
ResNet50 fine-tuned on PlantVillage, exported to ONNX and dynamically quantised to INT8 for
|
| 18 |
+
CPU-only serving. Part of [CropGuard](https://github.com/abhinav7289A/CropGuard), an
|
| 19 |
+
end-to-end MLOps pipeline.
|
| 20 |
+
|
| 21 |
+
## Results (held-out test set, n=8,125)
|
| 22 |
+
|
| 23 |
+
| Model | Accuracy | Macro-F1 | Note |
|
| 24 |
+
|---|---|---|---|
|
| 25 |
+
| fp32 | 0.9911 | 0.9865 | reference |
|
| 26 |
+
| INT8 (dynamic) | not evaluated | not evaluated | not served - see below |
|
| 27 |
+
|
| 28 |
+
Macro-F1 is the metric to read here, not accuracy: the dataset is imbalanced ~36x, so accuracy
|
| 29 |
+
is dominated by the largest classes.
|
| 30 |
+
|
| 31 |
+
**Only `cropguard.onnx` (fp32) is published.** Dynamic INT8 quantization was tried and is
|
| 32 |
+
*not* shipped: `quantize_dynamic` rewrites every `Conv` into `ConvInteger`, which ONNX
|
| 33 |
+
Runtime's CPU backend has no optimized kernel for. Measured on an Intel Alder Lake CPU it ran
|
| 34 |
+
at **1567 ms/image against 19 ms/image for fp32** - a 75x regression in exchange for 4x less
|
| 35 |
+
disk. Dynamic quantization suits MatMul-dominated models (Transformers, RNNs), not CNNs; the
|
| 36 |
+
correct approach for a Conv-heavy network is *static* quantization with a calibration set,
|
| 37 |
+
which emits the optimized `QLinearConv`. That is not built yet.
|
| 38 |
+
|
| 39 |
+
## The split is grouped by leaf, and that matters
|
| 40 |
+
|
| 41 |
+
PlantVillage contains 54,305 images of only ~7,600 *distinct physical leaves* -
|
| 42 |
+
roughly 7 photographs of each. A standard per-image stratified split scatters those
|
| 43 |
+
near-duplicates across train and test, so a model can score well by memorising leaf identity
|
| 44 |
+
rather than learning disease morphology. Measured on this dataset, a naive split leaves
|
| 45 |
+
**74.2% of test images sharing a leaf with training**.
|
| 46 |
+
|
| 47 |
+
This model was trained on a **leaf-grouped** split instead:
|
| 48 |
+
|
| 49 |
+
| | Naive stratified | Grouped (used here) |
|
| 50 |
+
|---|---|---|
|
| 51 |
+
| test images sharing a leaf with train | 74.2% | **0.0%** |
|
| 52 |
+
| train / val / test | - | 38,008 / 8,172 / 8,125 |
|
| 53 |
+
|
| 54 |
+
So the accuracy above is measured on a holdout with no leaf overlap. Note that it is *not*
|
| 55 |
+
much lower than typically published PlantVillage figures - the honest reading is that this
|
| 56 |
+
dataset is genuinely easy, not that leakage was inflating everything.
|
| 57 |
+
|
| 58 |
+
## Intended use
|
| 59 |
+
|
| 60 |
+
Identifying disease on **single leaves photographed against a plain background**, matching the
|
| 61 |
+
PlantVillage capture protocol.
|
| 62 |
+
|
| 63 |
+
## Limitations - read before deploying this
|
| 64 |
+
|
| 65 |
+
- **Lab images, not field images.** Every training image is a detached leaf on a uniform
|
| 66 |
+
background under controlled lighting. Real photographs from a farm - variable lighting,
|
| 67 |
+
occlusion, multiple leaves, soil backgrounds - are a different distribution, and published
|
| 68 |
+
work on this dataset reports large drops there. This model has **not** been evaluated on
|
| 69 |
+
field photographs.
|
| 70 |
+
- **Per-class metrics for rare classes are noisy.** Eight classes have fewer than 100 test
|
| 71 |
+
images; the smallest (`Potato___healthy`) has 24. A recall of 0.833 there is 4 mistakes, and
|
| 72 |
+
its confidence interval spans roughly +/-15 points. Do not read those per-class numbers as
|
| 73 |
+
precise.
|
| 74 |
+
- **Confidence is not calibrated.** Training used label smoothing (0.1), which deliberately
|
| 75 |
+
caps confidence, so predicted probabilities are expected to understate. No temperature
|
| 76 |
+
scaling has been fitted.
|
| 77 |
+
- **`uncertainty` is predictive entropy**, not epistemic uncertainty. It cannot distinguish an
|
| 78 |
+
ambiguous input from one far outside the training distribution - an out-of-distribution
|
| 79 |
+
image can produce confidently wrong output with low entropy.
|
| 80 |
+
- 38 classes across 14 crops only. Anything outside that set is silently forced into one of
|
| 81 |
+
them.
|
| 82 |
+
|
| 83 |
+
## Training
|
| 84 |
+
|
| 85 |
+
ResNet50 (timm, ImageNet-pretrained), 224x224, batch 64, AdamW (lr 3e-4, weight decay 1e-4),
|
| 86 |
+
cosine schedule, label smoothing 0.1, medium augmentation, 12 epochs, mixed precision.
|
| 87 |
+
Checkpoint selected on `val_f1_macro`.
|
| 88 |
+
|
| 89 |
+
## Usage
|
| 90 |
+
|
| 91 |
+
```python
|
| 92 |
+
import numpy as np, onnxruntime as ort
|
| 93 |
+
from huggingface_hub import hf_hub_download
|
| 94 |
+
|
| 95 |
+
path = hf_hub_download("XiElonMAsk/cropguard-models", "cropguard.onnx")
|
| 96 |
+
session = ort.InferenceSession(path, providers=["CPUExecutionProvider"])
|
| 97 |
+
|
| 98 |
+
# Preprocessing must match training: resize short side to 256, centre-crop 224,
|
| 99 |
+
# scale to [0,1], normalise with ImageNet mean/std, NCHW.
|
| 100 |
+
logits = session.run(["logits"], {"input": batch})[0]
|
| 101 |
+
```
|
| 102 |
+
|
| 103 |
+
`cropguard.serving.model_loader` in the repo implements exactly that preprocessing.
|
| 104 |
+
|
| 105 |
+
## Citation
|
| 106 |
+
|
| 107 |
+
Dataset: Mohanty, Hughes & Salathe (2016), *Using deep learning for image-based plant disease
|
| 108 |
+
detection*, Frontiers in Plant Science.
|