| ---
|
| license: cc-by-sa-4.0
|
| tags:
|
| - image-classification
|
| - agriculture
|
| - plant-disease
|
| - onnx
|
| datasets:
|
| - mohanty/PlantVillage
|
| metrics:
|
| - accuracy
|
| - f1
|
| ---
|
|
|
| # CropGuard - Crop Disease Classifier (38 classes)
|
|
|
| ResNet50 fine-tuned on PlantVillage, exported to ONNX and dynamically quantised to INT8 for
|
| CPU-only serving. Part of [CropGuard](https://github.com/abhinav7289A/CropGuard), an
|
| end-to-end MLOps pipeline.
|
|
|
| ## Results (held-out test set, n=8,125)
|
|
|
| | Model | Accuracy | Macro-F1 | Note |
|
| |---|---|---|---|
|
| | fp32 | 0.9911 | 0.9865 | reference |
|
| | INT8 (dynamic) | not evaluated | not evaluated | not served - see below |
|
|
|
| Macro-F1 is the metric to read here, not accuracy: the dataset is imbalanced ~36x, so accuracy
|
| is dominated by the largest classes.
|
|
|
| **`cropguard.onnx` (fp32) is the model that serves traffic.** `cropguard.static-int8.onnx` is
|
| published alongside it for comparison; the **dynamic** INT8 build is deliberately *not*
|
| published.
|
|
|
| Dynamic quantization was the wrong tool here. `quantize_dynamic` rewrites every `Conv` into
|
| `ConvInteger`, which ONNX Runtime's CPU backend has no optimized kernel for: measured on an
|
| Intel Alder Lake CPU it ran at **1567 ms/image against 19 ms for fp32**, a 75x regression in
|
| exchange for 4x less disk. It suits MatMul-dominated models (Transformers, RNNs), not CNNs.
|
|
|
| *Static* quantization with a calibration set emits the optimized `QLinearConv` instead, and is
|
| **78 ms/image** - 20x faster than dynamic and accuracy-neutral (0.9897 vs 0.9893 on a 3,000
|
| image subset, 9 disagreements). It is still 3.2x slower than fp32 on the machine it was
|
| measured on, and the reason is hardware rather than the graph: that CPU has no VNNI
|
| instructions, without which ONNX Runtime emulates each INT8 multiply-accumulate in several
|
| AVX2 instructions. Server CPUs usually do have VNNI, where the ranking may reverse - which is
|
| exactly why both files are here to be benchmarked on whatever hardware you are running.
|
|
|
| ## The split is grouped by leaf, and that matters
|
|
|
| PlantVillage contains 54,305 images of only ~7,600 *distinct physical leaves* -
|
| roughly 7 photographs of each. A standard per-image stratified split scatters those
|
| near-duplicates across train and test, so a model can score well by memorising leaf identity
|
| rather than learning disease morphology. Measured on this dataset, a naive split leaves
|
| **74.2% of test images sharing a leaf with training**.
|
|
|
| This model was trained on a **leaf-grouped** split instead:
|
|
|
| | | Naive stratified | Grouped (used here) |
|
| |---|---|---|
|
| | test images sharing a leaf with train | 74.2% | **0.0%** |
|
| | train / val / test | - | 38,008 / 8,172 / 8,125 |
|
|
|
| So the accuracy above is measured on a holdout with no leaf overlap. Note that it is *not*
|
| much lower than typically published PlantVillage figures - the honest reading is that this
|
| dataset is genuinely easy, not that leakage was inflating everything.
|
|
|
| ## Intended use
|
|
|
| Identifying disease on **single leaves photographed against a plain background**, matching the
|
| PlantVillage capture protocol.
|
|
|
| ## Limitations - read before deploying this
|
|
|
| - **Lab images, not field images.** Every training image is a detached leaf on a uniform
|
| background under controlled lighting. Real photographs from a farm - variable lighting,
|
| occlusion, multiple leaves, soil backgrounds - are a different distribution, and published
|
| work on this dataset reports large drops there. This model has **not** been evaluated on
|
| field photographs.
|
| - **Per-class metrics for rare classes are noisy.** Eight classes have fewer than 100 test
|
| images; the smallest (`Potato___healthy`) has 24. A recall of 0.833 there is 4 mistakes, and
|
| its confidence interval spans roughly +/-15 points. Do not read those per-class numbers as
|
| precise.
|
| - **The raw softmax is not calibrated.** Training used label smoothing (0.1), which caps
|
| achievable confidence at (1-eps) + eps/K = 0.9026 and leaves the model systematically
|
| *under*-confident. Temperature scaling fitted on the validation split (T = 0.591) cuts
|
| expected calibration error from 0.0895 to 0.0036 and is applied in the serving path, but the
|
| ONNX graph published here emits **logits** - apply the temperature yourself, or the
|
| probabilities you compute from it will understate.
|
| - **`uncertainty` is predictive entropy**, not epistemic uncertainty. It cannot distinguish an
|
| ambiguous input from one far outside the training distribution - an out-of-distribution
|
| image can produce confidently wrong output with low entropy.
|
| - 38 classes across 14 crops only. Anything outside that set is silently forced into one of
|
| them.
|
|
|
| ## Training
|
|
|
| ResNet50 (timm, ImageNet-pretrained), 224x224, batch 64, AdamW (lr 3e-4, weight decay 1e-4),
|
| cosine schedule, label smoothing 0.1, medium augmentation, 12 epochs, mixed precision.
|
| Checkpoint selected on `val_f1_macro`.
|
|
|
| ## Usage
|
|
|
| ```python
|
| import numpy as np, onnxruntime as ort
|
| from huggingface_hub import hf_hub_download
|
|
|
| path = hf_hub_download("XiElonMAsk/cropguard-models", "cropguard.onnx")
|
| session = ort.InferenceSession(path, providers=["CPUExecutionProvider"])
|
|
|
| # Preprocessing must match training: resize short side to 256, centre-crop 224,
|
| # scale to [0,1], normalise with ImageNet mean/std, NCHW.
|
| logits = session.run(["logits"], {"input": batch})[0]
|
| ```
|
|
|
| `cropguard.serving.model_loader` in the repo implements exactly that preprocessing.
|
|
|
| ## Citation
|
|
|
| Dataset: Mohanty, Hughes & Salathe (2016), *Using deep learning for image-based plant disease
|
| detection*, Frontiers in Plant Science.
|
|
|
